🛠️ Steven Gong

Search

SearchSearch

Aug 31, 2024, 6 min read

Hard Problems

This a non-exhaustive list of hard engineering problems I’ve run into during my lifetime, and how I’ve approached them. Apply Problem-Solving skills.

Elon Musk’s favourite question to ask during an interview:

  • “The people that really solved the problem, they know exactly how they solved it. They know the little details. The people that pretended to solve the problem, they can maybe go one level, and then they get stuck.”

Why do I want to work on hard problems? Because it is Important Work. Because it’s fun. Because it makes life worth moving.

What makes an engineering problem hard?

There are some general themes to hard engineering problems:

  1. Complexity: Lots of moving parts, hard to narrow down problem
  2. Resource Constraints: Optimizing within set of constraints in uncertainty
  3. Need for Innovation: No current solution exists, need to completely invent something new

I need to Ship.

Hard Problems I want to work on

I want to be hired to work on these hard problems. Shoot me an email if you’d want me contributing to these.

Autonomous Driving at Scale

  • Why this problem is hard: Handling edge cases. The long tail.

Humanoids / Manipulators / Robots at scale

  • Why this problem is hard: Robotics has trouble generalizing at the moment. It doesn’t work in new scenarios.

Drone Racing

  • Why this problem is hard: Racing drones move at very fast speeds. Being able to process all this data and make decisions in real-time at milimeter level precision takes an immense level of engineering.

Hard Engineering Problems I’ve worked on

Writing Visual SLAM from scratch for a VR Headset (Summer 2024) I worked on this project for a few months to get a deep understanding of how Visual SLAM works under the hood.

  • Video on YouTube here, GitHub repo here
  • Why this problem is hard:
  • How I approached it: Wrote it from scratch in Python (switched over from C++) to make debugging easier, using basic libraries like OpenCV for feature detection and matching, and g2o for the optimization (else I’d never finish the project on my own). Use dataset (TUM RGB-D dataset) to get ground truth data and incrementally improve SLAM system
  • Project logs found in VR Headset Logs

Poker AI (Summer 2024) This project was sparked out of my interest for Poker (No-Limit Texas Hold’Em). I wondered what it took to build a superhuman Poker AI, which led me down to this rabbit hole.

  • Video on YouTube here, GitHub repo here
  • Why this problem is hard:
    • EXTREMELY Large search space in imperfect information. Classical search that is used to solve Go and Chess doesn’t work. Value function is not clear.
    • Goal here is to find a Nash equilibrium. So you solve it on an abstracted game (smaller version). But even the abstracted game is very big.
    • Poker is a sequential imperfect information game
    • Implementing the rules of poker is a little tedious, but not too hard.
  • How I approached it: Read up on lots of research papers and implement them. Solving Poker is equivalently to find a strategy that achieves Nash Equilibrium. This is computed using CFR. However, solving Poker is too big, so I need to abstract it. 2 abstractions:
    • Card abstraction - cluster cards using equity distributions
    • bet abstraction - only allow certain bet sizes

NVIDIA (Fall 2023) Got to pursue my aspirations to work on robots through the NVIDIA Isaac ROS team for an internship. Helped out with testing ROS packages on Nova Carter, and built a camera-only racing robot which runs various packages from the Isaac ROS stack on a resource-constrained environment. Learned a lot about programming for performance: intra-process vs. inter-process comms, process vs. threads, schedulers, memory allocators, node composition in ROS2, etc.

  • Why this problem is hard: Resource constraints (8GB of ram). Running out of memory and out of compute. Real-time requirements
  • How I approached it: Used NVIDIA’s Isaac ROS Visual SLAM + Isaac ROS ESS to do the mapping, path planning and real-time occupancy detection the camera. Profiled the system and reduce memory usage by using dynamic instead of fixed memory allocation.

Siemens / Enlighted (Spring 2023) I wrote software for a Create3 robot that was used to collect bluetooth signal strengths in a room.

  • Why this problem is hard: Essentially needed to design a robot that would generate a path such that a robot sweeps every part of some area with one of its sensors (Coverage Planning).
  • How I approached it: Essentially implement a line sweep algorithm that does the coverage planning. Set waypoints, and use Nav2 to tell the robot to navigate to those waypoints.

F1TENTH (Autonomous 1/10 RC Car) Made a video series with 1M+ views on YouTube that you can watch here.

  • Why this problem is hard: Mostly an engineering challenge. but was my first time using ROS on a physical robot, and I barely knew C++ at the time. Had to use various packages like SLAM (used slam_toolbox), particle filters. Also, had to generate racing lines, and writing my own controllers to follow racing lines, and generate occupancy grids to dynamically avoid obstacles.
  • How I approached it: Slowly built the stack up over months. Incrementally improve the system.
  • See F1TENTH Logs

Ericsson (Summer 2022) Was part of a research team that was looking to use ML to automate antenna calibration. The problem was formulated at finding a phase offset that could maximize signal strength.

  • Why this problem is hard: It’s basically formulated as a MAB problem, where the goal is to find the optimal phase offset to maximize signal strength. This is essentially solving the Exploration and Exploitation problem.
  • How I approached it: Experimented with various MAB algorithms, like Thompson Sampling and UCB, diligently document the various differences on the same given dataset.

Competitive Programming (Summer 2022) I spent an entire summer of 2022 grinding Codeforces after work as I had nothing else to do in Kanata, Ottawa.

  • Technical skills required: Coming up with solutions fast, need to think very fast and strong reasoning capabilities. Hard because you need to be fast and accurate

WATonomous (Self-Driving Car) My school’s self driving car design team. I led perception.

  • Why this problem is hard: Complex system
  • How I approached it: Build one thing at a time

University of Waterloo

CS341 was the toughest, because it’s about learning to figuring out a solution, but proving why your solution is correct, on the fly.

Misc Problems

Sports

Badminton: how do you unlearn muscle memory?

Learning to become a good programmer

A 10x Developer isn’t just about pumping out 10,000 lines of code a day. It’s about making his code useful to others. For all these projects, I slowly gain insight into why design decisions made. What I acquire is nuance.

Graph View

Backlinks