Remote Operation Interface for Heavy Machinery — Unreal Engine & C++
A remote-operation station whose entire operator software, live IP camera feeds,CAN-bus telemetry and joystick control, runs as one C++ application in Unreal Engine.
Outcome
- Public live demo at Bauma 2022
- Real HMK 635 WL remote operation
- Live machine mode + Unreal simulation mode
- One software stack for camera feeds, telemetry, HMI and control input
System architecture
- Machine side: IP cameras + CAN-bus telemetry through a network gateway
- Station side: Unreal Engine + C++ application
- HMI: engine-native UI, not embedded web views
- Input: industrial joysticks and pedals encoded into machine control messages
- Secondary mode: same interface connected to an Unreal simulation
Construction sites are among the last places you want a human to be when something goes wrong. Hidromek's answer was Opera: a remote-operation station that moves the operator out of the machine entirely a sculpted pod with a seat, industrial joysticks and dual displays, designed by the Hidromek Design Studio and first shown as the award-winning Vision Opera concept.
My role was to make the pod work. I was the sole developer of the station's operator software: a C++ application built on Unreal Engine that receives the machine's camera streams and telemetry, renders the interface, and turns joystick input into machine commands. Industrial design, mechanical build and the machine-side engineering were the work of my Hidromek colleagues; everything running on the station's screens was mine.
The station and the machine are two ends of one network link. On the machine an HMK 635 WL wheel loader cameras and the CAN bus feed a gateway that streams video and telemetry to the station over TCP/IP. On the station, a single Unreal Engine application does the rest: it decodes the camera feeds, binds live CAN data to an HMI layer rendered natively in-engine, and encodes joystick and pedal input into control messages travelling back up the same link.
The interface itself started as Figma screens. Rather than embedding web views, I rebuilt it as engine-native UI the only way to keep rendering, video and input inside one low-latency loop, and to reuse the same interface unchanged in simulation.
How do you stream live camera feeds into Unreal Engine?
Treat cameras as textures, not as windows. Each IP camera stream from the machine is pulled over the network, decoded, and written into a dynamic texture inside the engine, where it can be composited and laid out like any other material: a forward driving view on the upper display, and the machine's 360° top-down surround view on the lower one. Latency is the whole game in teleoperation every stage of the pipeline, from stream ingest to final frame, exists to keep the operator's hands and the machine in sync.
Opera has a second mode. With no machine connected, the same application runs a full Unreal Engine simulation of a Hidromek wheel loader physics, work equipment, terrain behind the identical interface and controls. In the studio this made the station a development and training rig; at fairs it meant the pod could demonstrate itself even without a machine. One codebase, two realities: the live machine and the simulated one are interchangeable behind the same screens.
Opera made its public debut at Bauma 2022 in Munich, the world's largest construction machinery fair, where the station remotely operated a real HMK 635 WL wheel loader on the Hidromek stand a demo independently captured by trade press and equipment channels. The design programme behind it, Vision Opera, earned a German Design Award (2022) and an Automotive Brand Contest award (2021), and Hidromek's engineering team has since published openly on the operator-assistance systems built around the station (Karahan, IConTES 2024). The videos below are Hidromek's public materials.
Role & credits. Operator software (Unreal Engine, C++), camera and telemetry integration, HMI implementation and the simulation mode: Kadir Özkan. Industrial design of the Opera pod: Hidromek Design Studio. Machine-side engineering, autonomy and operator-assistance systems: Hidromek engineering teams.
Opera set the pattern most of my work has followed since: a physical product, a real-time software core and a human interface treated as one system the same approach behind my later automotive HMI and interactive product work. More about that path on the About page; the architecture behind this case is broken down further in the Lab.