Gamestream

I like the idea of handhelds like the Steam Deck and have been thinking about getting one for a while, but whenever I try to justify it I can’t come up with much beyond wanting to play games in bed or on the go, which never feels like enough of a reason to buy another device.

I also own a PS5 with PlayStation Plus Premium that I rarely use, and I knew the PlayStation Portal¹ was a thing, but I wasn’t sure it was worth $200 when I’d just bought a pretty big phone with a great screen, so I wanted to see if I could somehow use that for games instead. I’d used GeForce Now before and knew about projects like Moonlight and Sunshine², so I thought it might be possible, and it was! There was coincidentally already another app that did this natively on iOS, but I didn’t want to pay for their subscription, so I started building my own.

Signing in

Signing in was the first mess. Nothing on Sony’s side is documented, so I started investigating it by watching what the official PlayStation Plus app and the Portal do when they log in. This ended up being a chain of codes, tokens and handshakes that all have to happen in the right order, with response keys that change depending on which part of Sony you’re talking to. I also spent longer than I’d like to admit working out why Sony’s login page opened fine in Safari but refused to load inside my own app.

Once that worked, the other half was making sure you only ever have to do it once, so the app keeps your session in the Keychain and uses it to get whatever fresh codes a launch needs instead of sending you back to a login screen every time you open it.

Getting a picture

Sony’s cloud streams over Takion³, the same protocol family PS4 and PS5 Remote Play use, which meant chiaki-ng, an open-source Remote Play client, already knew how to do most of the hard parts, and what I had to add was everything between signing in and handing it a server to talk to.

That still took a while, because once I had everything connected the server just went silent, and it stayed silent through a couple of rounds of guessing until I captured the Portal’s traffic mid-stream and saw that every packet it sent had four extra bytes at the front that nothing I’d read had mentioned. With those added, the first real frame I got back was Big Walk’s welcome screen (which I recommend you go play, it’s amazing).

Audio came next and was broken by a single byte, because a setting I’d changed for controller feedback also changed how audio packets were read, so the whole stream was off by one. Haptics were a similar story. I’d convinced myself the cloud just didn’t send them, until it turned out they’d been arriving the whole time and I was feeding them to the audio decoder, because the app asked for a different version of the protocol than the Portal does, and only the Portal’s version sends haptics as their own stream, along with adaptive triggers and the rest of the DualSense feedback.

PS5, PS4 and PS3

PS5 games were the first to work, and PS4 and PS3 each run on their own system, with almost nothing carrying over between them. PS4 games refused to launch until I found they needed a different entitlement ID from the one I was sending. PS3 needs its own kind of sign-in code, tops out at 720p, and uses an older color standard for its video, so during my whole first PS3 session every color was slightly off, which was most obvious on skin.

Streaming from your own PS5 is a different thing again, since that goes through Remote Play instead of the cloud, so the app asks PSN for a session, gets a connection through to the console, and only then hands over to chiaki the same way a cloud launch does. When you’re done it puts the console back to sleep, like Sony’s own app does, so it isn’t left sitting there awake with nothing connected.

Making it feel like a PlayStation

For the UI I looked a lot at the Backbone app⁴, the PS Store on an actual PlayStation, and the Portal, because I wanted it to feel right and look kind of like a normal PlayStation UI.

Most of that comes down to the controller. With your phone clipped into one you can’t really reach the screen, and iOS doesn’t give you a way to move focus around with a controller like it does on Apple TV, so I built that part myself, with the d-pad and sticks moving through rows of games, the shoulder buttons switching between tabs, and everything still working normally with touch.

In addition to this, I added sound effects and haptics to the navigation to make it feel more refined. Every move and button press gives its own feedback through the controller, depending on the action, and a click out of the speaker. The clicks are generated in the app instead of recorded, with a few slight variations of each so that holding the stick through a row of games doesn’t play the exact same sound over and over.

Then there’s everything around the games themselves, like a detail view with the store information, showing which platforms a game can stream on, filtering and collections for when the library gets big, and the overlays during a stream, which include a PS button and let the phone screen act as the DualSense touchpad.

I also wanted it to be the one place you need, without having to go anywhere else to find something new to play. If a game is part of PlayStation Plus but isn’t in your library yet, you can still press Play, and the app checks the store, asks if you want to add it, and then starts it right away. Without that, the whole thing felt like just another step in a process instead of an actual end-to-end experience.

Where it’s at

Right now I’m the only one using Gamestream, and it’s been really fun. It works great, and I do actually end up playing in bed and on the go now. I haven’t decided whether I want to fully release it yet. There’s still a good amount to flesh out first, like managing your PS5 and sending game downloads to it straight from the app.