Hey HN, We're Jash and Mahimana, cofounders of Flywheel AI (https://useflywheel.ai). We’re building a remote teleop and autonomous stack for excavators.

Here's a video: https://www.youtube.com/watch?v=zCNmNm3lQGk.

Interfacing with existing excavators for enabling remote teleop (or autonomy) is hard. Unlike cars which use drive-by-wire technology, most of the millions of excavators are fully hydraulic machines. The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints. This means excavators mostly do not have an electronic component to control the joints. We solve this by mechanically actuating the joysticks and pedals inside the excavators.

We do this with retrofits which work on any excavator model/make, enabling us to augment existing machines. By enabling remote teleoperation, we are able to increase site safety, productivity and also cost efficiency.

Teleoperation by the operators enables us to prepare training data for autonomy. In robotics, training data comprises observation and action. While images and videos are abundant on the internet, egocentric (PoV) observation and action data is extremely scarce, and it is this scarcity that is holding back scaling robot learning policies.

Flywheel solves this by preparing the training data coming from our remote teleop-enabled excavators which we have already deployed. And we do this with very minimal hardware setup and resources.

During our time in YC, we did 25-30 iterations of sensor stack and placement permutations/combinations, and model hyperparams variations. We called this “evolution of the physical form of our retrofit”. Eventually, we landed on our current evolution and have successfully been able to train some levels of autonomy with only a few hours of training data.

The big takeaway was how much more important data is than optimizing hyperparams of the model. So today, we’re open sourcing 100hrs of excavator dataset that we collected using Flywheel systems on real construction sites. This is in partnership with Frodobots.ai.

Dataset: https://huggingface.co/datasets/FlywheelAI/excavator-dataset

Machine/retrofit details:

  Volvo EC380 (38 ton excavator)
  4xcamera (25fps)
  25 hz expert operator’s action data

We’re just getting started. We have good amounts of variations in daylight, weather, tasks, and would be adding more hours of data and also converting to lerobot format soon. We’re doing this so people like you and me can try out training models on real world data which is very, very hard to get.

So please checkout the dataset here and feel free to download and use however you like. We would love for people to do things with it! I’ll be around in the thread and look forward to comments and feedback from the community!

> The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints

I've actually spent a decent amount of time running an excavator, as my Dad owns a construction / road building company. It was a great summer job!

An important note about the pilot hydraulics is that they _provide feedback to the operator_. I would encourage any system that moves these controls on behalf of a remote human operator or AI to add strain gauges or some other way to measure this force feedback so that this data isn't lost.

The handful of "drive by wire" pieces of equipment that my Dad or other skilled operators in my family have ran were universally panned, because the operators are isolated from this feedback and have a harder time telling when the machine is struggling or when their inputs are not sufficiently smooth. In the automotive world, skilled drivers have similar complaints about fully electronic steering or braking systems, as opposed to traditional vacuum or hydraulic boosting approaches where your foot still has a direct hydraulic connection to the brake pads.

What a fantastic business. I've watched lots of digger (as we call them here) drivers in action and I'm in awe of what these machines can do and how incredibly skilled their operators are. We have several large rock walls on our property and watching these guys delicately picking and placing huge rocks to key them in just blows my mind.

There was a YouTube video recently of an AI-assisted digger making a wall out of the concrete rubble from a demolition.

I believe the applications for really smart excavators must be huge. Sounds like this might be a step on that voyage.

This looks very interesting. I understand that hydraulic retrofit is compelling, but you may want to consider CANbus-based retrofits as well (similar to how CommaAI works, where it simply sends messages to the CANbus network to control steering or speed just like ADAS features do). Modern equipment is moving away from hydraulic controls toward a fully digital cockpit and being able to plug in and teleop (for less than the cost of the manufacturers expensive robotics suite) would be amazing.

I have a Cat 289D skid steer I would happily contribute to the effort if you guys move into the compact equipment space (compact being a relative term, as my machine is only 6 tons compared to your 38 ton machine)

Would you be able to replicate this with the heavy equipment and movements needed to plug orphaned oil wells? In Texas alone, it's a TAM of ~$38B, and ~$150B for the entire US.

The Looming Disaster Under America's Biggest Oil Field [video] - https://news.ycombinator.com/item?id=45361022 - September 2025

Texas has thousands of abandoned oil and gas wells. Who is responsible for cleaning them up? - https://www.texastribune.org/2025/05/08/texas-orphan-wells-e... - May 8th, 2025

The Rising Cost of the Oil Industry’s Slow Death - https://www.propublica.org/article/the-rising-cost-of-the-oi... - February 22nd, 2024

Well plugging SOP:

https://www.epa.gov/natural-gas-star-program/well-plugging

https://www.osha.gov/etools/oil-and-gas/abandoning-well

Would be really cool if you could set up a pilot on https://www.youtube.com/@dustupstexas

I have a question about competition. I had a friend who was in mining about a decade ago. He told me that their mines were fully autonomous back then -- that the equipment managed itself and sent back sensor data.

Is that different than what you are doing now?

Whats the budget for getting an MVP for heavy equipment ?

I ask in all seriousness since, for example, retrofitting regular semis to electric requires millions and millions just to get started

This scratches the itch I've had since playing with Tonka trucks in the back yard when I was 5... Are you hiring?

How much safety training have you done with the models? i.e. does it know to stop if it's about to drive over a human?

Congratulations! Looks like an interesting project!