🛠️ Steven Gong

Search

SearchSearch

Aug 04, 2025, 6 min read

Research

  • The goal of research is to create new knowledge in the world.
  • The goal of engineering is to solve problems in the world.

My goal since entering university was to become the best engineer. In the past year, it’s starting to transition to becoming the best researcher in robot learning.

Elon says researchers and engineers are the same things.

Important links:

I have a mixed opinion on doing research. However, I’ve realized that working on the state-of-the-art requires me to read quite a few research papers, both for WATonomous and my personal project for the Poker AI.

See F1TENTH Research Proposal

Doing Interesting Research (from talking with Leroy)

if you want to do interesting research, you should think on your own, and come up with your own approach.

if you just google the existing approaches, then you are just going to become overwhelmed. and that is not good. it’s not a great way to approach these.

Don’t do research for the sake of doing research. Because there are really important problems to solve.

Okay, I actually want to pump out a real research paper.

I need to conduct goal-driven research as opposed to idea-driven research.

Reflection

I think also finding collaborators is really going to exponentiate the rate at which you are learning. Work together, but I mean you learn less, but you are going to achieve more.

Knowing that you cannot learn everything in this world is very disappointing. But once you realize you cannot learn everything in this world, it is also relieving. Because you can focus on the things that are interesting to you.

My problem is that I was trying to solve a problem that is too large. You need to find a scope, a focus.

Other

Serendipity: It’s not how to solve a particular problem, it’s what problem to solve. Andrej Karpathy talks about this here “an exercise in the outer loop”, and Hitchhikers Guide to the Galaxy, where Elon Musk talks about this.

Reading to get solutions does not seem to be the way to do great research.

  • If you read all the time what other people have done, you will think the way they thought
  • If you want to think new thoughts that are different, get the problem reasonably clear and refuse to look at any answers until you’ve thought the problem through carefully how you would do it, how you could slightly change the problem to be the correct one

Research is about framing what question to ask.

Andrej Karpathy on Meta-Problem: “You spend most of your time on the outer loop, figuring out what problems are worth solving and what problems are ripe for solving. You’re constantly imagining yourself solving hypothetical problems and asking yourself where that puts you, what it could unlock, or if anyone cares. If you’re like me this can sometimes drive you a little crazy because you’re spending long hours working on things and you’re not even sure if they are the correct things to work on or if a solution exists.”

Surveying Research

Learnings from Yash Pant look at what problems people are solving.

My Opinion on Research

I think research is amazing because people are working together to solve unsolved problems, these are some of the hardest problems that exist in society.

I don’t like research because it is too specialized, you need to spend a year or 2 or 3 or 4 or 10 to really crack one niche topic. I would much rather jump around and experience a bunch of things. However, I need to make sure I have a good focus.

  • This is not true, just think really hard about a problem.

Research is Tunnel Vision.

2023-01-13: I think my views on research has drastically changed today. It is not about the solution, it’s about the Meta-Problem: what problems are we even trying to solve that are important?

You signed up for this. Getting lost and confused. And the more you learn, the more you realize how much you don’t know.

Implementing Papers

How to implement a paper:

Basically, before you try to implement an advanced paper, maybe try an easier version of that paper. you won’t have as much headache with it. There are a few things I would like to implement

Reading and Understanding Research Papers

How to read research papers:

Another one:

How to Read Research Papers byAndrew Ng:

  1. Compile list of papers (+ medium / blog posts)
  2. Skip around list (and eliminate the ones that are not relevant)
    1. Some numbers, 5-20 papers you get an okay understanding, 50-100 papers and you’ll master that subject

Personal tips:

Publish a Paper / Doing Research

This stanford guide seems very good: https://cs231n.github.io/choose-project/

How to read a paper

Multiple passes

  1. Title/abstract/figures (summarizes the whole paper)
  2. Intro + Conclusion + Figures + skim rest
  3. Read but skip/skim math
  4. Read whole thing, but skip parts that don’t make sense (to prioritize your time)

Try to keep these things in mind:

  • What did authors try to accomplish?
  • What were the key elements of the approach?
  • What can I use myself?
  • What other references do I want to follow?

Sources of papers:

  • Twitter
  • ML subreddit
  • NIPS/ICML/ICLR
  • Friends

Tips for understanding Math

  • Rederive from scratch

Code

  • Run open-source code
  • Reimplement from scratch

Steady reading, not short bursts.

To read actively reading, ask yourself the following questions:

  • Do I understand all the terminologies? Do I need to do some background reading to help me understand?
  • Can I explain the research question and why it is important?
  • Can I summarize the methods and the main findings?
  • Does the author’s interpretation persuade me?
  • Is the research credible? Is the journal trustworthy?

Professors

Heard this from Pieter Abbeel, which I find a little sad since he talks about how he doesn’t write as much code anymore.

But Code is Truth. So he is losing touch. But I guess that is everyone.

Andrej Karpathy on Research

http://karpathy.github.io/2016/09/07/phd/

You and your research http://karpathy.github.io/2016/09/07/phd/

Graph View

Backlinks