Hey Future Tim, in case you ever need to set it up again, here is a guide to setting up game streaming from a beefy desktop PC (game host / backend) to a Raspberry Pi 5 (game display client / frontend) with low latency, both for the input devices (keyboard/mouse or controllers linked to the Pi) and the returning audio/video stream. There are a few reasons to do this, but in my case it’s simply that during pandemic lockdowns my desktop PC, along with its nice GPU, got swallowed into a “home office”, otherwise known as a dingy corner of the basement, and due the realities of life after children, will probably stay that way. For gaming though, I’d still much rather do it from the comfort of the living room couch on a big TV (i.e. “PC couch gaming”), and ideally get the kiddos involved too. I already have a Raspberry Pi 5 set up there as a low-power fileserver and movie streaming box, can I use it to play games from a Real Computer too?
Short answer: Yes, totally, and with no noticeable lag or framerate issues (if you’re not married to 4K or wifi). This setup uses the open-source Moonlight streaming client on the Pi and the open-source Sunshine server on the PC. I couldn’t find much hard information about performance on the RPI5 when I set this up, so decided to write up what I learned here.
The Sunshine/Moonlight combo is a universal open-source replacement for NVidia’s now-EOL’ed GameStream technology and similar schemes such as Steam Link, but one which works across several major GPU brands (AMD, Intel, NVidia). This setup is basically a modern VNC or remote-desktop application souped-up suped-up with realtime hardware video compression (H.265, HEVC or similar) and a focus on high framerates at low latency for gaming.
Upfront caveats and limitations:
- Again, this is basically modernized VNC. While there is some lip-service toward starting a specific game or app (e.g. Steam client) upon connection, your PC desktop is easily accessible and displayed/interactable while things get going, or if the game crashes, etc. In other words, this (alone) will not keep the kids out of your … tax return collection, and while you can use it over the internet with decent performance (I’m told), I wouldn’t share it with anyone I didn’t extremely trust.
- Many usual “remote desktop” limitations apply (for Windows anyway): you must be logged in already, and Sunshine streams the currently logged-in user’s apps/desktop. E.g. a freshly booted PC sitting at the Windows login screen is not connectable, and you can’t set up a dedicated ‘gaming user’ to run as with just the games (and not access to your desktop/files), at least without fiddliness like running the whole shebang in a VM. If the user is logged in but the screen is merely locked, you can connect but need to unlock the screen (w/ password) first. Proposed workaround for “unconnectable” in Windows is setting the host up to auto-login, then immediately lock the desktop.
Performance Tweaks:
- For smooth framerates (60+Hz, no dropped frames) from a RPi5, the only configuration setting I really found to matter is dropping the Raspberry Pi 5’s resolution (out to the TV) to 1920×1080. In my testing, higher resolutions including 4k worked, but the framerate dropped to ~30Hz (Moonlight reported 50% or more “frames dropped due to network jitter”, more on that below) and was noticeably a bit choppy. Again, this is purely a function of the Pi’s output resolution (Preferences -> Screen Configuration), and has nothing to do with any Moonlight settings (e.g. PC-side resolution or framerate), assuming the setting there isn’t causing the PC itself to struggle. Obviously, commanding higher resolution from the PC than the Pi will display won’t have any benefit, but 4K vs 1080p coming in from the PC seems to have no performance impact.
- If you simply must run your Pi at 4K resolution, try setting the remote PC’s framerate (Moonlight configuration screen) to double the TV framerate, e.g. 120Hz for a 60Hz display, if the PC will tolerate it. Here, frames are being dropped due to coming in ‘too late’ to render (due mainly to delay getting to the end of the Pi’s video pipeline at 4k); kicking up the PC framerate helps ensure at least one remote frame per local frame (60Hz) will come in close enough to the local ‘due date’ to satisfy the target framerate, even if many of those 120Hz frames are still being dropped.
- For reliably smooth framerates, use a wired connection (ethernet cable) to both ends instead of WiFi if possible. The Pi5 has gigabit Ethernet right onboard, and chances are your existing “WiFi” access point has a few sockets’ worth too. In initial testing with an 802.11n connection at reasonable range, it was generally smooth at recommended data rates but with occasional large hiccups, which went away with a wired connection.
- Beware of limitations in the displayed statistics when tweaking for performance; for example, the delivered framerate will drop when little/no screen motion is occurring; this is an optimization rather than a performance artifact. And see caveats on ‘network jitter’ messages below.
Some other hints to get the most out of this setup:
Virtual / Dummy displays and Dummy Plugs
For remote gaming streamed to a display in a different room, there’s a good chance the host PC is running headless or its physical monitor is turned off. Depending on the monitor, the PC will not be able to detect its supported resolutions while off, causing Sunshine/Moonlight to fail or fall back to a failsafe resolution like 1024×768. In this case, you can fake an always-connected monitor to report usable resolutions, either by installing a “dummy plug” (fake HDMI dongle that reports plausible resolutions but doesn’t actually display anything, available from the usual web shops), or a dummy display driver (software-only solution). Some driver options I know of are:
- https://github.com/itsmikethetech/Virtual-Display-Driver
- https://www.amyuni.com/forum/viewtopic.php?t=3030
These options may also be useful if you want to stream to an HDR-capable TV, but the PC-attached monitor doesn’t support it.
Using External Bluetooth Dongles
Yes, I am using a BLE keyboard and mouse for this. Purists may wring their hands about latency, but I haven’t noticed a meaningful difference vs. wired or proprietary RF dongles for my dirty casual tastes. If you use e.g. Switch joycons wireless, you’re probably used to it anyway.
In my experience so far, the Bluetooth reception on the RPI5 is a little weedy, especially if a bulky heatsink has been attached that sits near the antenna and the board is shoved in amongst the usual metal-filled set-top boxes and consoles in your TV hutch (guilty as charged on both counts). Combine with ‘couch distance away’ and Bluetooth keyboard/mouse designed to be used within finger’s reach of the computer, and it may not work.
An easy fix for this is an external Bluetooth module plugged into a USB port, and possibly on a short extender cable to get it out of the metal mess a bit. To go this route, you just need to install a different Bluetooth manager to control device pairing across multiple interfaces. I used the ‘blueman’ package and the resulting ‘blueman-applet’ desktop configuration widget, as the stock one (which doesn’t provide its name) seems to assume a single Bluetooth interface, and I didn’t want to fiddle with trying to nerf the builtin one.
Game launchers
Steam (specifically, Big Picture Mode) seems to be the default “6-foot interface” for choosing and launching individual games. However, other game launchers can be used (e.g. to provide a unified interface to non-Steam games). Playnite is a popular open-source option. Sunshine (on the PC end) provides some support for customizing multiple launcher/game options and pre/post launch scripts (e.g. resolution tweaks).
Detailed Performance Details and Profiling
You can send Ctrl+Alt+Shift+S on the Pi side to enable a diagnostic overlay giving framerate and other statistics. Using this, I did some testing and found:
- The RPI5 really struggles to deliver >1080 @ 60Hz, and this is the main setting to tweak for performance (configured via RPI OS).
- PC resolution (configured via Moonlight) has no noticeable impact on delivered framerate (640×480 vs. 4K)
- Data rate (0.5Mbps ~ 150MBps, configured via Moonlight) has no noticeable impact on delivered framerate. Compression artifacts will become noticeable at lower data rates of course, but the Pi does not struggle with the maximum rate.
The following tables were taken with the following setup unless otherwise noted:
- Client: Raspberry Pi 5 running Bookworm ‘version who-knows’; Pi is adequately heatsinked to prevent thermal throttling.
- Moonlight 6.0.1, Sunshine v0.23.1
- RPI and host PC at 1920×1080@60Hz, HEVC encoding
- Datarate: 60Mbps
- V-Sync and Frame Pacing enabled
- Host game: Portal 1 (a quite old game to rule out any host PC framerate issues).
The below table shows the effect of Pi screen resolution on Moonlight statistics. Note the Rendering Frame Rate, ‘Frames dropped due to network jitter’, and especially ‘Average frame queue delay’. This is with a quick spot-check with sample size of 1, so some natural variation is expected.
| Pi Resolution | 1920×1080 | 3840×2160 | 4096×2160 |
| Incoming/Decoding Frame Rate | 60.22 | 60.07 | 60.01 |
| Rendering Frame Rate | 60.22 | 40.37 | 30.0 |
| Host Processing Latency min/max/avg (ms) | 3.3/5.1/3.8 | 3.6/41.2/5.0 | 3.5/42.7/6.0 |
| Frames dropped due to network jitter | 0.00% | 32.79% | 43.44% |
| Average network latency (variance) (ms) | 2 (4) | 1 (0) | 1 (0) |
| Average decoding time (ms) | 3.94 | 5.80 | 5.89 |
| Average frame queue delay (ms) | 0.02 | 39.56 | 36.43 |
| Average rendering time (including monitor V-sync latency) (ms) | 0.86 | 4.00 | 0.80 |
Before continuing, be aware that the ‘Frames dropped due to network jitter’ is a bit of a misnomer; it actually represents “the percentage of frames dropped because they are too early or too late to render” (missed deadlines), due to delay at network or any later point in the rendering process (i.e. not specific to network jitter). In this case, at 60Hz we expect to render a new frame every 16msec, but with ~40msec worth of frame queue delay (time between when frame is decoded and when it is rendered to the display), many of those will miss their deadline and be dropped.
The next table compares host (incoming) resolution effects on framerate. The values are limited by my PC monitor which does not quiiite do 4K. While there are some changes, recall the critical threshold of 16msec per frame at 60Hz; we stay comfortably below that on average and no frames are dropped.
| Host Resolution | 1920×1080 | 3840×2160 |
| Incoming/Decoding Frame Rate | 60.22 | 59.93 |
| Rendering Frame Rate | 60.22 | 59.93 |
| Host Processing Latency min/max/avg (ms) | 3.3/5.1/3.8 | 9.1/17.5/9.8 |
| Frames dropped due to network jitter | 0.00% | 0.00% |
| Average network latency (variance) (ms) | 2 (4) | 3 (4) |
| Average decoding time (ms) | 3.94 | 12.07 |
| Average frame queue delay (ms) | 0.02 | 8.60 |
| Average rendering time (including monitor V-sync latency) (ms) | 0.86 | 2.02 |
Leave a Reply