Javascript Has Gamified Programming

Modern Javascript has gamified programming by creating more micro-success moments (aha! finally got it!).

But we know there is No Silver Bullet[1], so it couldn’t have created more moments where you finally solve your hard problems (“essential complexity”). Instead, it has created many opportunities to beat problems you wouldn’t have in another environment (“accidental complexity”) and made it fun to beat them.

All these small annoyances create a mounting pressure in the Javascript programmer to do anything but work on front-end code, resulting among other things in the Cambrian explosion of development tools that modern Javascript is so famous for, and which is also the source of most of the accidental complexity. Tooling annoys programmers, causing them to create tooling that annoys programmers… A positive feedback loop!

Another way this pressure releases is by writing technical blog posts. Which is to say, I have an important deadline for tomorrow morning. OK, OK, getting back to work.

[1] https://en.wikipedia.org/wiki/No_Silver_Bullet, probably the most important software engineering essay you’ll ever read

Advertisements

I challenge you to a game of Sokoban Golf!

Graphics of my programmable Sokoban game which this was the prototype for, by Fiverr user applemoment
Graphics of my programmable Sokoban game which this was the prototype for, by Fiverr user applemoment

Before I dived in earnest into writing the new version of my computer game that teaches Python, this time with game mechanics based on the classical puzzle game “Sokoban”, I wanted to make sure that Programmable Sokoban is a fun concept. So I wrote a prototype in a couple of hours.

The programmers among you will enjoy it.

“Golf” is a game played by programmers, where you have to solve a problem in as few keystrokes as possible. I challenge you to solve these Programmable Sokoban levels in as few keystrokes as possible!

Cover Letter

 

Warning: another technical post.The duckrabbitI wrote this cover letter many years ago, when I wanted to get accepted to the Matzov unit of the Israeli Defense Force (in Israeli, every male is conscripted for three years at age 18, and I wanted to put my skills to good use.)

Like the picture above, it can be interpreted two different ways. You could read it as a letter in the English language explaining why my skillset is a good fit Matzov… or you could read it as a program in the Brainfuck programming language (can you tell what it does?) This is made possible because most characters in the English alphabet are considered comments in Brainfuck. Still, writing it required pulling an awful lot of tricks.

On the right you can see a size-optimized hand-coded 8086 assembly language Brainfuck interpreter, and instructions on how to execute it. While writing the letter took a few hours, writing the interpreter took three days without sleep. Shaving the last byte off took almost half a day.

Matzov Cover

Click for full version.

Notice how periods are meaningful commands in Brainfuck, so there’s a question of whether the periods on the right are or aren’t part of the program. In practice, it doesn’t matter because if they were, they would be after an infinite loop and never actually execute.

Edit: Hi HN. At your request, here‘s a .txt I extracted from the .pdf.

Watch behind my shoulder: Debugging

Warning: long-winded tech post. If you don’t want to improve your debugging skills or don’t have lots of free time, don’t read this.

I decided to do some pro-bono debugging, and as I was diving in I thought that some of you might be interested in a glimpse of how I work.

I find that when debugging complex issues, keeping a “lab manual” and documenting every single action you take before you take it, although tedious and boring, gets the problem solved much faster. This time I’ll open my manual.

I will document what I think and what I do. I won’t document general debugging theory and techniques, as that would interrupt my flow too much (and probably make this into a book). I’ll just say this:

When debugging, you should throw all trust you have in yourself out the window. You start knowing that you’re wrong  (there is a bug) despite being confident you’ve done every single thing right, so thinking “I can see the problem is not here” without proving it by experiment is almost thought-crime and will waste you hours and sanity.

Background

I’m considering submitting a talk proposal for PyCon 2014. They have a page with proposal resources. Unfortunately, I cannot click any links on the page. Normally I’d just go do something else, preferring not to waste time on websites that hurt so much to use. Since this is an open source web site for a project that has a lot of value for me, I figured why not exercise those debugging muscles and help them out with the issue.

Exploration

I click a few times, trying different things and observing the outcomes. Middle click – my habit – fails. Ctrl+left click – another way to “open in new tab” in Firefox – fails. Left click succeeds, but this doesn’t solve the issue to my satisfaction. I like new pages to open in new tabs. I see the same behaviour for different links – HTTP and HTTPS links, same-site and other-site links. The browser seems to register my clicks, in the sense that the link is marked as focused after my mouse leaves it.

Is the issue with the site or my browser? I click the FAQ link on a Paul Graham essay that I have open. It opens the link in a new tab. I click again in us.pycon.org – still nothing. The issue is with the site.

Is the issue with the page or the site? I left-click to another page on the menu (turns out middle clicking a menu item closes the menu but doesn’t open the link) and try links there – still nothing. The issue is with the entire site.

Is the issue with my browser or all browsers? I initially think I don’t care enough to install Chrome, but having to write it in my manual it smells wrong. Many scenarios where knowing the answer will narrow my search pop into my mind, and I’d rather spend a few minutes solving some issues with my Arch Linux installations so I can install Chrome than spending a few hours mulling over the PyCon issue.

I’ll have to update my OS first, so in the meantime lets continue. I’ll add TODO: Chrome to the end of this document, keep it after the cursor and remove it when I check, so I don’t forget this.

Is it an HTML issue? A quick right-click-inspect-element shows perfectly normal HTML, so probably not:

<a href="/2014/dashboard/">account dashboard!</a>
<a href="http://us.pycon.org/2014/speaking/proposal_advice">Proposal Advice</a>

What does the site Javascript look like? I open Firebug “script” tab. There’s a little but of inline JS, an http://us.pycon.org/2014/site_media/static/cache/js/ea388955cc03.js that probably cobbles together all  page-specific JS, jquery.min.js, bootstrap.min.js, and theme.js. jQuery is 1.8.3, bootstrap is 2.3.1.

theme.js is short and has a call to jQuery ajaxSend that looks very suspect, a quick google explains that it sets a hook that runs every time an AJAX request is sent, in this case it seems like the hook adds a CSRF token. Is it responsible? Easy to check: Add a breakpoint in the hook, click a link. The breakpoint is not invoked. Nothing to see here, moving on.

What happens when I middle click a link

In Firebug’s “Script” tab, I click the “pause play” icon – the great “break on next” feature. Nothing happens. Good, no periodic JS to confuse me. Then I click a link. It pauses in minified jQuery, in line 4 of:

I try again, this time stopping in line 5 of:

I try a few more times, getting the same two lines.

Actually, at first it always stopped in the one and only line of jquery.min.js. I googled and found Javascript Deminifier, a Firefox extention that prettified the source for me (variables still have one-letter names, but I’ll figure that out later if needed.)

I suddenly remember that I have NoScript installed (although disabled for the site), which tends to cause strange JS problems. Is it responsible for the bug? In a different browser profile with no NoScript, still no middle click.

OK, this would be a good time to stop what I’m doing and search for existing bug reports in jQuery. Who knows, I might be trying to solve a solved problem. I find references to problems with jQuery’s live() function, later deprecated in favour of deleteage() and on(), but I can only find a few uses of delegate() and breakpoints on their hooks don’t trigger when I middle click.

I find a page indicating that I should probably look for return False, .preventDefault() or .stopPropagation() as culprits. I search the code for them, ignoring results in jQuery.min.js and resisting the urge to ignore results in bootstrap.min.js (despite my instinct that well-tested frameworks are bug-free, it might be a Bootstrap behaviour. Note to self: try other Bootstrap sites with the same version).

I find a few uses of said functions, set breakpoints and middle click a link. Break! This might be it. The debugger stopped in bootstrap.min.js, in a piece of code that looks very unashamed:

Is this intended Bootstrap behaviour? Googling for bootstrap “on(click.dropdown-menu)” turns out this page that tells me that what I’m looking for is bug 7118 in Bootstrap versions 2.3.1 and 2.3.2 (bingo! That’s the version the site uses!).

So how should they fix it? Upgrade Bootstrap? According to the drupal bug, 2.3.2 – the last release before 3.0 – suffers from the issue. According to the Bootstrap issue tracker, it seems that it was fixed:

Skud commented June 11, 2013

Has this actually been rolled into a bootstrap release yet?

@Skud: #7614 was declined. b9bd82b was rolled into v2.3.2 to fix #7118. Unfortunately, that fix introduced another bug: #7968.

According to my own testing, it doesn’t work in 2.3.1 and does in 2.3.2.

So PyCon webmaster, please upgrade to Bootstrap 2.3.2!

Epilogue

Well, this was a bit anticlimatic. I was hoping for an interesting Javascript bug and found a reported Bootstrap issue. However, this is still a good example of how I debug complex issues in code I don’t understand well (in this example, someone else’s code).

I found the problem and isolated it to a specific line without understanding what the huge codebase does. I didn’t click “step into” even once. I documented everything I did and it made me follow good paths that I wouldn’t have followed otherwise. I resisted my self-confidence many times and one of them proved critical. All in all, I consider this a good learning resource on debugging.

I — I don’t even see the code. All I see is blonde, brunette, redhead…

I was holding up on posting until I can say more about PythonForAll (it’s not that I didn’t get out of the building, just that it’s becoming a bit more serious so I can’t publish things before they are final) but I just have to write about this:

matrix-code

For a long time I’ve wanted to code a screensaver showing matrix code, just like the gajillion clones you can already find on the internet, but instead of showing random characters, this one would capture (“sniff”) network packets and show them in encoded form.

The theory is that if you watch this long enough, you’ll begin to “see through the code” – at first, you’ll learn to decode the characters into numbers, and at more advanced stages you’ll learn to parse the network protocols intuitively. I have many reasons to believe this:

  • I’ve seen people learning languages and musical instruments (and experienced it myself)
  • I’ve heard that one of the Samba developers can decode raw NBNS packets in his head (one of the most horrible encodings I’ve seen, see here 1.3.1)
  • I knew someone who could synchronize a dial-up modem with his voice
All images, characters, content and text are copyrighted and trademarks of J.D. Frazer except where other ownership applies. Don’t do bad things, we have lawyers.

 

Announcing Wirematrix

Today I saw this HN thread and was reminded of this idea. I decided that the time had come. A night of work later, an alpha version exists and the main feature works well:

If you'll look closely, you'll see that some patterns can be noticed immediately even from one static screenshot
If you’ll look closely, you’ll see that some patterns can be noticed immediately even from one static screenshot

You can get it on github. If you are not experienced with compiling linux software, I advise you to wait for a more user friendly release. Drop a comment saying you’re interested, it will motivate me to create one.

There’s work to do on friendliness of installation and on shininess of visualization. Other than that it’s complete. Python and Cairo rock!

Becoming a Programmer

Continuation and contrapunct to Should you Learn To Code?

There are many paths you can walk to become a programmer. If you bear with me, I will try to sketch a few below. But first, let me give you a few navigational tips…

Know This

If I must tell you one blatant lie about learning that will serve you well in everything you do, it is this:

Walk a path that you enjoy every step of.

I like to learn playing musical instruments almost as much as I like playing them. Watching myself repeating the same motions again and again and suddenly become fluent in them, observing how, when not touching an instrument for a month and returning to it, I play better than when I left, the experience is a most amazing window to how the mind works.

There is one parameter by which I choose which instruments to take up:

I should enjoy the sound I make when playing it completely untrained.

If, when I just begin, I can already sit for hours and listen hypnotized to my own playing (or, in the first few days, noise), then I will stay long enough to improve. I’ll just never quit because I’m enjoying myself so much.

The same goes for everything you learn. Find a resource you enjoy learning from, and you’ll never quit. If you find yourself bored while learning, somebody else probably teaches it (or something similar) in a way you’ll enjoy more.

The Best Learning Environment

I had the great luck to study programming together with a group of friends in the same non-obligatory class. Every day at least one of us would do something awesome (relative to what we knew how to do), show it to everyone else, and we’d all run home and try to improve on it or outdo it.

I attribute mainly to this great learning environment that I fell in love with programming so strong and fourteen years later am practicing it professionally. Every member of our group was still programming last time I heard from him.

Find friends to learn with. Show each other stuff you did, try to improve on what others did, try to figure stuff out together. Play. It’s the best possible drive. Even if you’re a loner at heart like me.

If you need, bribe your friends to start learning with you (but not so hard that there is any doubt whether they are still friends). Invite them once a week for a night of programming, pizza and drinks. Go out of your way to be a good host and make those nights the best party in town. Go out of your way to make learning fun for all of you. The pizza and time spent will cost you much less than university tuition and you’ll gain much more.

Sketching A Few Paths

A Path In Red

He likes to hang out online, and he hears that today’s adventurers are doing startups, so he figures he’s gotta learn how this whole web startup thing goes. He asks on some forum how to make an MMORPG, they wisely explain to him that it’s the wrong question to ask before you have 10-20 years of programming experience but he can learn web development easily.

So he picks up Rails, maybe follows a written tutorial or a few videos on PeepCode and decides to build a forum software for his favourite boards because he can’t stand how the admins don’t allow signatures larger than 1024×768. He has a demo pretty quickly to show to his friends with lots of stuff you can click, and the praises keep him driven to try some other stuff (he got bored with the forum software when it dawned on him how much logic there is to write that does absolutely boring things).

Eventually he decides to get together with some friends and build the browser turn based strategy game to beat all browser turn based strategy games. When he’s done with this or bored with it, he finds that by now he knows quite a lot about this web-dev stuff and he can actually get a job doing it, so he does, and at work he learns good practices.

Now he’s become really good at this, he’s done a bit of everything you can do in the web, he knows a bit of graphic design (and knows CSS inside-out) from the days when he built stuff for himself, he figures he’s ready for his first Web 2.0 startup.

A Path In Blue

He likes to build robots, and though at first programming them in some awful graphical programming environment seems sane to him for lack of knowing better, eventually he needs more computing power than the chips powered by graphical languages can give.

So he gets an Arduino and installs the IDE for it. Learning the weird dialect of C that powers this particular controller is a very daunting experience, but he persists only by his strong love for robots and his deep need to make them do more awesome things than what he could before.

Eventually, he wants to figure out something like path-finding algorithms or conputer vision. He downloads some libraries, maybe gets a book, plays with it a bit, and to really do what he wants he’s forced to learn how to really use C to its fullest. Maybe (poor guy) he even has to learn some C++.

He finds that this whole AI thing is actually very interesting. So by now he moves to simulations of robotics on a PC, which allow him to use saner dialects of C, and he may even hear that Ruby and Python exist, with which he can simulate things with much less effort (but he’s already gained the technological courage that comes from starting in a very low level environment where you need to tackel the bits once in a while, he knows what’s going on down there and he’s not afraid to dive in if there is a need).

He gets more and more into academic stuff, DSP, Computer Graphics, algorithms, cryptography, all of which he finds extremely elegant, and eventually decides to get his CS Ph.D. and work as an algorithm researcher for a big high tech firm.

Side note: That DSP link is to one of the best textbooks I’ve ever seen, and though it doesn’t deal with programming and almost doesn’t talk about it, it might turn out to be a great way to get into programming, because what it teaches is such an awesome application of it.

A Path In White

He enrolls to CS in university because it sounds nice, without ever having looked into it to see what it means.

At university they teach him to program, and some other stuff which is more relevant to CS research, and he figures pretty fast that all that mathy stuff is not for him.

So he graduates, gets a job, and at work they teach him how to program. Hopefully he gets his job in a shop full of smart professionals passionate about their occupation and not in a code monkey farm, otherwise he would be in grave danger of never knowing better.

A Path In Black

He spends most of his time playing computer games.

One day, one of his friends shows him a cool program that can “find” cheats for games. You just tell it “now I have 96 life points”, play for a while, tell it “now I have 77”, play, tell it “now I have 31”, and it tells you, “Found”. Now you can press a button in the program and have infinite life points.

Usually, every game has cheat codes the developers put there. But this program can let you create new cheats that perhaps the developer didn’t want to put there. He thinks this is pretty neat, and being a very curious person, asks his friend how it works.

His friend doesn’t know, so he asks online. Someone explains to him that a game’s memory is full of numbers, and one of those  numbers represents his life points. So if the cheat program knows where it is stored, it can simply write whatever number it wants there. To find that number, the program remembers all memory cells that have the number 96, later drops all those that didn’t change to 77, then drops all those who didn’t change to 31, and now it is left with only one cell, so that must be the correct one!

Armed with a new way of thought, he ventures further and finds that he can change the monsters’ stats in the game by changing a text file in the game’s directory. He figures the images must also be stored somewhere nearby so he asks again online and they explain to him that they’re in a proprietary archive format and give him a program he can use to edit it.

He is now interested in cheat programs not because of the amusing in-game effects, but because of the technical pleasure of venturing into a game’s internals and figuring out how it works and how it can be changed to your whim.

But he all he knows is how to use tools other people created and how to do simple changes, like finding text files with game content and changing them. He asks the guy who wrote his favourite tool how to learn more interesting techniques, and gets send to learn reverse engineering.

The tutorials teach him how, by understanding the way code is represented in EXE files, he can crack copy protection mechanisms. He quickly understands how the same information is relevant to cheating in games. Just like he learned to find the “conditional jump” that decides whether he gets a nag-screen about not having bought the software and change it to never show that nag-screen, he can find the “conditional jump” that decides whether he did super-damage and change it to always do it.

Now he’s drinking this knowledge up. He spends all his days doing reversing tutorials, doing “crackmes” – reverse engineering exercises, and creating cheats by reversing and patching games.

One time he even has a dream in bits, bytes, and DWORDs.

Along the way, he learns how to program so he can release cheat tools with nice GUIs. It comes very easily because he’d spent months reading other people’s code by reversing it, so he already thinks in code, he just had to learn the tools.

One day he learns about buffer overflows. With the same knowledge of how programs are executed, he learns a technique that lets him patch the software to do what he wants just by crafting input to the runnnig program, without any need to edit its EXE file! But more importantly, the technique is technically amazing! He finds there is nothing he enjoys more than to deconstruct a program so he can craft bits in the shape that will trigger a specific program to eat his input as new code to run, against it’s creator’s intentions. (Seriously, I need at least two hours and a blackboard to explain it, but this is the awesomest thing in the world. A truly amazing experience. Unfortunately, I personally never found time to practice it.)

So he finds buffer overflows (and more advanced kinds of vulnerabilities, like heap overflows) in software, writes tools to exploit them and publishes the tools to his friends on the scene, which use them to hack into computers. He reads a bit about hacking and figures that it’s like what he used to do with games, except with other kinds of systems, like other people’s computers and web servers, so he gets into that a bit, using his own tools and his friends’ to cheat servers around the world to do whatever he wants.

Eventually he’s out of high school and needs a job, so he either uses his computer hacking skills to perform online crime (or sell tools to those who do for loads of money) or gets a job in the security sector finding holes in software so his company can report them to the company who wrote it (and earns loads of money… as far as I know security is the highest paying sector of high tech).

A Path In Phosphorous Green

There was one computer at school and it cost $50K. The only things you could do on it were play Pong and program in BASIC. So after he got bored with Pong, he learned how to program it by studying the Pong code and trying to make changes and seeing what works and what doesn’t.

Then he found a book in the library about programming, but it was for a different brand of computer. He gulped it up cover to cover and then spent his days in school writing programs in his notebooks, for a computer that he didn’t have access to, with no way of knowing wether they’d have bugs in them or even run at all.

After school he’d stay in class to have time with the computer.

His teachers eventually figured out he gets excited by computers, so they got him a book about programming the computer that was in school, so he learned that. Only later he was told that programming in Assembly language, what the book taught, is supposed to be much harder and only meant for grown-up professionals.

He’d stay in school every day until dinner, writing small Assembly language programs, later learning from a magazine how to draw things on the screen fast enough for it to seem animated and creating a few games.

From here on, everything else he did with computers was much easier than how he began, so he progressed quickly. I’ll spare the descriptions because this post is becoming too long already and you can find a ton of these stories here for example.

Today he’s one of the masters. Those I aspire to become like, and mourn not having started with something as hard which would’ve taught me much better habits and given me much more courage to face hard problems. On the other hand, I’m not sure, had I taken such a hard path, if I’d still be programming or if I would’ve given it up.

Today, the closest there is to taking this path (except for getting a Commodore 64 for $25 on eBay and learning to program it) is something like this or this. I’ll repeat: I believe (with only anecdotes to prove me right) that learning the lowest-level programming first makes you a much better programmer, but it is much harder and less “visually” rewarding (it will take you the longest to have something pretty on screen) than other paths and may kill your drive to learn it.

So Choose Your Own Path

Among these and many others, there might be a path you’ll enjoy every step of. Walk with a confident step!

Should you learn to code?

Should you learn to read and write?

There is a discussion on the boards revolving around the eternal question of whether “non-techie” startup founders will put their time to good use by learning to code.

Five years ago, there was still some room for discussion. These days, I believe discussing it is a waste of time:

Yes. Learn to code.

As simple as that. But perhaps it is not as obvious to you as it is to me. Worry not! I will explain, give examples and even recommend where to start.

Since this is very relevant to my interests, I will create shorter and shorter version of this essay as time passes, incorporating your feedback, hoping to eventually converge on a version that passes the message. Continue reading