[Notes] Guides by Andrej Karpathy

Some tips from two insightful guides by Andrej Karpathy.

Doing well in your courses

TIPS

All-nighters are not worth it.

Attend tutorials or review sessions.

Considering the big picture and organization is the key. Create schedule of study, even if you don’t stick to it. For me this usually involves getting an idea of everything I need to know and explicitly writing it down in terms of bullet points. Consider all points carefully and think about how long it will take you to get them down. If you don’t do this, there is a tendency to spend too much time on beginning of material and then skim through the (most important) later material due to lack of time.

Always try to look at previous tests BEFORE starting to study.

Reading and understanding IS NOT the same as replicating the content. Make it a point to make sure that you can actually write down the most important bits, and that you can re-derive them at will.

Always try to collaborate with others, but near the end.

Don’t only hang out only with stronger students. Weaker students will have you explain things to them and you will find that teaching the material helps A LOT with understanding.

Go to the prof before final exam at least once for office hours. Even if you have no questions (make something up!) Profs will sometimes be willing to say more about a test in 1on1 basis.

Study well in advance. The brain really needs time to absorb material. Things that looked hard become easier with time. You want to alocate ~3 days for midterms, ~6 days for exams.

For things like math: Exercise > Reading.

Make yourself cheat sheet. What you want is to cram the entire course on 1 or more pages that you can in the end tile in front of you and say with high degree of confidence “This is exactly everything I must know”

Study in places where other people study as well, even if not the same thing.

Study very intensely RIGHT before the test.

VERY Important Advice

Undergrads tend to have tunnel vision about their classes. They want to get good grades, etc. The crucial fact to realize is that no one will care about your grades, unless they are bad. For example, I always used to say that the smartest student will get 85% in all of his courses. This way, you end up with somewhere around 4.0 score, but you did not over-study, and you did not under-study.

Your time is a precious, limited resource. Get to a point where you don’t screw up on a test and then switch your attention to much more important endeavors. What are they?

Getting actual, real-world experience, working on real code base, projects or problems outside of silly course exercises is extremely imporant. Professors/People who know you and can write you a good reference letter saying that you have initiative, passion and drive are extremely important. Are you thinking of applying to jobs? Get a summer internship. Are you thinking of pursuing graduate school? Get research experience! Sign up for whatever programs your school offers. Or reach out to a professor/graduate student asking to get involved on a research project you like. This might work if they think you’re driven and motivated enough. Do not underestimate the importance of this: A well-known professor who writes in their recommendation letter that you are driven, motivated and independent thinker completely dwarfs anything else, especially petty things like grades. It also helps a lot if you squeeze in at least one paper before you apply. Also, you should be aware that the biggest pet peeve from their side are over-excited undergrad students who sign up for a project, meet a few times, ask many questions, and then suddenly give up and disappear after all that time investment from the graduate student’s or professor’s side. Do not be this person (it damages your reputation) and do not give any indication that you might be.

Other than research projects, get involved with some group of people on side projects or better, start your own from scratch. Contribute to Open Source, make/improve a library. Get out there and create (or help create) something cool. Document it well. Blog about it. These are the things people will care about a few years down the road. Your grades? They are an annoyance you have to deal with along the way. Use your time well and good luck.

A Survival Guide to a PhD

Don’t play the game.
Finally, I’d like to challenge you to think of a PhD as more than just a sequence of papers. You’re not a paper writer. You’re a member of a research community and your goal is to push the field forward. Papers are one common way of doing that but I would encourage you to look beyond the established academic game. Think for yourself and from first principles. Do things others don’t do but should. Step off the treadmill that has been put before you. I tried to do some of this myself throughout my PhD. This blog is an example - it allows me communicate things that wouldn’t ordinarily go into papers. The ImageNet human reference experiments are an example - I felt strongly that it was important for the field to know the ballpark human accuracy on ILSVRC so I took a few weeks off and evaluated it. The academic search tools (e.g. arxiv-sanity) are an example - I felt continuously frustrated by the inefficiency of finding papers in the literature and I released and maintain the site in hopes that it can be useful to others. Teaching CS231n twice is an example - I put much more effort into it than is rationally advisable for a PhD student who should be doing research, but I felt that the field was held back if people couldn’t efficiently learn about the topic and enter. A lot of my PhD endeavors have likely come at a cost in standard academic metrics (e.g. h-index, or number of publications in top venues) but I did them anyway, I would do it the same way again, and here I am encouraging others to as well. To add a pitch of salt and wash down the ideology a bit, based on several past discussions with my friends and colleagues I know that this view is contentious and that many would disagree.

Closing thoughts
I can’t find the quote anymore but I heard Sam Altman of YC say that there are no shortcuts or cheats when it comes to building a startup. You can’t expect to win in the long run by somehow gaming the system or putting up false appearances. I think that the same applies in academia. Ultimately you’re trying to do good research and push the field forward and if you try to game any of the proxy metrics you won’t be successful in the long run. This is especially so because academia is in fact surprisingly small and highly interconnected, so anything shady you try to do to pad your academic resume (e.g. self-citing a lot, publishing the same idea multiple times with small remixes, resubmitting the same rejected paper over and over again with no changes, conveniently trying to leave out some baselines etc.) will eventually catch up with you and you will not be successful.

So at the end of the day it’s quite simple. Do good work, communicate it properly, people will notice and good things will happen. Have a fun ride!