Every now and then your web application does something so wild and unpredictable that it crashes the browser that you’re running it on. In order to create a better product for our users, we would need to log pertinent information every time our app crashes. Unfortunately there is no way to send a crash log before or during the crash due to the unpredictable nature of the crash and the browser’s web environment no longer working. The best thing to do is to send the logs after the crash. This post will go through a technique to detect when the user’s previous session has crashed so that we can perform the relevant logging actions.

Preface

Currently I’ve only been able to support this technique on Google’s Chrome and Mozilla’s Firefox browser. This technique takes advantage of these two browser’s support for restorable sessionStorage. The WHATWG HTML spec currently describes sessionStorage to delete itself once the browser context ends but also makes a note that browsing contexts can continue to persist even after the browser is closed. This is useful if the browser chooses to save the session to reload it after the user closes the browser window or if the browser window crashes. WHATWG spec on sessionStorage

Simulating the Crash

First, we’ll have to find out ways to simulate crashes on our browsers. Fortunately for our sake, its pretty easy to do for both Mozilla and Chrome.

Crashing on Chrome

1. Go to the page you want to crash
2. Paste this into your url ‘chrome://crash’
3. Press Enter

That was pretty easy. This brings up Chrome’s ‘Aw Snap!’ error page which simulates as if an irrecoverable error actually happened on the previous site you were on.

Crashing on Firefox

1. Go to the page you want to crash
2. Open your Task Manager on Windows or Activity Monitor on OSX
3. Find your firefox process and end the process or force quit it

Force stopping the processes trigger’s Firefox’s crash conditions and emulates a real crash scenario. Firefox should ask you to restore your previous tabs along with your previous session as well.

Go ahead, try it!

Now that we know how to crash this page, go ahead and try it! When you come back there should be an alert letting you know that you came back from a crash and at what time

Acting on the Crash

Now that we know how to crash the browsers we just need to place the code in.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

   window.addEventListener('load', function () {
      sessionStorage.setItem('good_exit', 'pending');
      setInterval(function () {
         sessionStorage.setItem('time_before_crash', new Date().toString());
      }, 1000);
   });

   window.addEventListener('beforeunload', function () {
      sessionStorage.setItem('good_exit', 'true');
   });

   if(sessionStorage.getItem('good_exit') &&
      sessionStorage.getItem('good_exit') !== 'true') {
      /*
         insert crash logging code here
     */
      alert('Hey, welcome back from your crash, looks like you crashed on: ' + sessionStorage.getItem('time_before_crash'));
   }

The code above is checking if the user successfully closes, refreshes, or browses to another page by applying a good_exit = true flag in our sessionStorage. When the user successfully loads the page, the ‘good_exit’ flag will be set to ‘pending’ as we’re not sure if the user will successfully exit from this session. If the user unsuccessfully closes, refreshes, or browsers to another page and manages to crash the browser, the ‘good_exit’ flag will not be changed to ‘true’ and will stay as ‘pending. Everytime the user loads the page we check whether they’re coming back from a crash. If they are, then we apply our crashalytics code.

Things to do after a crash

Remember that your sessionStorage is saved and recovered so anything that your user was doing before the crash can be recovered. This means you can store the last visited url, the time that they visited your site, any actions they have taken on the page, anything that you can store in sessionStorage can be recovered.

Say for instance the last thing the user was doing before the crash was typing a very long response in a text box. You can save the contents of the text box every few seconds into sessionStorage so that you can reload it once the user comes back from the crash. Or if you just want some analytics on the last actions the user took you can create a sessionStorage item to log and trace the actions taken by the user by listening to clicks and key inputs on your document. When the visitor returns you can post that information back to your server so you have something to go on when trying to fix your buggy site. Or when a user comes back to your page you can directly ask them to submit a bug report about what they were doing before they crashed. Many native desktop programs already do something like this and now you too can bring this feature to your buggy web application!

Caveats

An already listed caveat is this may or may not currently work for Internet Explorer or Safari. I wasn’t able to reliable crash those browsers to test whether they bring back sessions after crashes. Another caveat is that since sessionStorage content isn’t shared between tabs, users will have to refresh or reload the same tab with the same sessionStorage in order for the crash logging code to execute. If users open a new tab to connect after a crash then there will be no record of there being a crash. It will act as if you started a new browser session. As you can see this method isn’t 100% reliable, but it does give options to cover some of your user base.

tested on Chrome version 43.0.2357.81 and Firefox 38.0.5 on June 21, 2015

Preface: Unfortunately, the JSX code highlighting may not be perfect. Each JSX React code example is followed by the transformed version to JavaScript, followed by the actual rendered code in the page. Feel free to inspect the elements to get a better idea of how React renders your code.

You may have landed here because you tried to render a list in React and wondered why it didn’t work. You may have tried passing and rendering the list as a prop and wondered why your list is still all inline. You may feel this way if you came from an Angular background where rendering with ng-repeat was largely left to magic. React gives you a finer level of control that makes using it more easier to understand.

The following example shows what happens when simply rendering a list in React without any processing. Inspect the code and notice how all the elements are displayed in a single list item.

Bad Example

JSX

1
2
3
4
5
6
7
8
9
10

  var IncorrectListRender = React.createClass({
    render: function() {
      return (
        <ul>
          <li>{this.props.list}</li>
        </ul>
      )
    }
  });
  React.render(<IncorrectListRender list={[1,2,3,4,5]} />, document.getElementById('incorrect-list-render'));

Compiled to JavaScript

1
2
3
4
5
6
7
8
9
10

  var IncorrectListRender = React.createClass({displayName: "IncorrectListRender",
    render: function() {
      return (
        React.createElement("ul", null,
          React.createElement("li", null, this.props.list)
        )
      )
    }
  });
  React.render(React.createElement(IncorrectListRender, {list: [1,2,3,4,5]}), document.getElementById('incorrect-list-render'));

Rendered Result

The above example doesn’t cut it.

You may also have tried to individually pick out elements in the array and plant them in your render code. This would work, but its not very flexible. Notice how the second render call with the longer list is only able to display the first five elements.

Bad Example

JSX

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

  var InflexibleListRender = React.createClass({
    render: function() {
      return (
        <ul>
          <li>{this.props.list[0]}</li>
          <li>{this.props.list[1]}</li>
          <li>{this.props.list[2]}</li>
          <li>{this.props.list[3]}</li>
          <li>{this.props.list[4]}</li>
        </ul>
      )
    }
  });
  React.render(<InflexibleListRender list={[1,2,3,4,5]} />, document.getElementById('inflexible-list-render1'));
  React.render(<InflexibleListRender list={[1,2,3,4,5,6,7,8,9,10]} />, document.getElementById('inflexible-list-render2'));

Compiled to JavaScript

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

 var InflexibleListRender = React.createClass({displayName: "InflexibleListRender",
    render: function() {
      return (
        React.createElement("ul", null,
          React.createElement("li", null, this.props.list[0]),
          React.createElement("li", null, this.props.list[1]),
          React.createElement("li", null, this.props.list[2]),
          React.createElement("li", null, this.props.list[3]),
          React.createElement("li", null, this.props.list[4])
        )
      )
    }
  });
  React.render(React.createElement(InflexibleListRender, {list: [1,2,3,4,5]}), document.getElementById('inflexible-list-render1'));
  React.render(React.createElement(InflexibleListRender, {list: [1,2,3,4,5,6,7,8,9,10]}), document.getElementById('inflexible-list-render2'));

Rendered Result

A good solution would be flexible to the amount of items in a list and would render with its own containing element in a repeating fashion. Notice how the following render function is able to accomodate arrays of all sizes. This is due to rendering in a list of React Elements created from the list of numbers coming in from the props.

Good Example

JSX

1
2
3
4
5
6
7
8
9
10
11
12
13

  var ProperListRender = React.createClass({
    render: function() {
      return (
        <ul>
          {this.props.list.map(function(listValue){
            return <li>{listValue}</li>;
          })}
        </ul>
      )
    }
  });
  React.render(<ProperListRender list={[1,2,3,4,5]} />, document.getElementById('proper-list-render1'));
  React.render(<ProperListRender list={[1,2,3,4,5,6,7,8,9,10]} />, document.getElementById('proper-list-render2'));

Compiled to JavaScript

1
2
3
4
5
6
7
8
9
10
11
12
13

  var ProperListRender = React.createClass({displayName: "ProperListRender",
    render: function() {
      return (
        React.createElement("ul", null,
          this.props.list.map(function(listValue){
            return React.createElement("li", null, listValue);
          })
        )
      )
    }
  });
  React.render(React.createElement(ProperListRender, {list: [1,2,3,4,5]}), document.getElementById('proper-list-render1'));
  React.render(React.createElement(ProperListRender, {list: [1,2,3,4,5,6,7,8,9,10]}), document.getElementById('proper-list-render2'));

Rendered Result

The solution above is flexible enough to accomodate as many or as little list items that comes from the props field.

Lets break it down to see why it works

First each React class object needs to have a render function. Think of the render like a screen refresh. The render function is supposed to return a value or list of values (most commonly composed of React Elements) that will be drawn onto the screen. The render function is automatically called whenever the object’s state is signalled to change. States are outside the scope of this guide but you can read more about state here.

Notice how HTML is plainly inserted into a JavaScript function. The JSX transformer/compiler will automatically detect HTML and React tags inside JavaScript and convert them to the equivalent JavaScript expression. If you would like to read more about the end result feel free to try out the compiler and read up on what it does to your tags.

Also take note of the curly brackets peppered through out the HTML tags. The brackets signify to the JSX transformer that there is actual JavaScript inside of it. Combining JSX and JavaScript together allows us to use the ease of HTML tagging and the logical power of JavaScript to create dynamic HTML templates right inside of our React code. In the case of the list example, note how we start with a containing <ul>…</ul> block. This </ul> block is then split with a pair of brackets that perform a mapping to create and return a new list of React Elements <li>{listValue}</li>. Also notice again how {listValue} had to be surrounded again in brackets since its embedded within HTML elements. If you want to know more about what you can do with React Elements take a look here.

Conclusion

The React renderer is actually quite flexible. You don’t necessarily have to feed it just Numbers or just React Elements but can mix and match all sorts of JavaScript objects. You can filter and parse the original objects in your list and use JavaScript logic to render a completely new list with dynamic React Elements. You can even plug in your own Custom React components in your render function as well, but you already knew that didn’t you. Truly understanding how your JSX code will be compiled to plain old JavaScript is the key to writing predictable React everytime.

preface: It seems like this is the second time I’ve plugged Hired, I ensure you reader that I am not in their books.

Hired is just like a typical job hunting site, except this time the roles are reversed; the recruiters hunt while you respond. Pretty crazy concept right? Instead of having you search for positions through Monster, Dice, or other job listing sites, you publicly advertise yourself and your abilities and the companies come to you.

The Story

I’ve been around six to seven months unemployed. During that time I’ve been working on personal projects to familiarize myself with the current trends in web development. I finally felt that I was ready to contribute with my new-found skills and was ready to go on the hunt. I had some companies in mind, the typical engineering bigs, Amazon, Microsoft, Facebook so on and a few smaller ones. I also wanted to see what else was out there and decided to try out Hired.

The Rundown

I sign up, set up my profile, and list my skills and locations I want to work. In the same week I get a call from my ‘Talent Advocate’ to walk me through the whole process, the kind of work I’m looking for, and some advice on how to make my profile stand out to employers. It was essentially a recruiting phone screen that you can’t mess up. After the screen I fix up my profile a little, add a nice picture, list out my GitHub projects, and make my location preference much more specific. My Advocate then places a one-liner describing me and my work that companies will view when browsing over my profile. After that I’m ready to roll and submit my profile to be available for the next ‘auction’ which starts every Sunday each week and lasts for two weeks.

Hired Roadmap

1. 1. Fill out your profile
2. 2. Get a call from a ‘Talent Advocate’ or Hired recruiter
3. 3. Fix up your profile
4. 4. Put it up for the next ‘auction’
5. 5. Wait for the ‘offers’ to roll in

Now when Hired says candidates usually get 5-15 ‘offers’, they really just mean introductions but we knew that already right? In fact from my experience, the offered equity and salary and signing bonuses are still negotiable for the better or worse. These offers are just a way for companies to entice you so you can start talking. In fact Hired requires you to get back to each and every company that contacts you. You have the option of declining the offer outright, re-negotiating terms or clarifying job descriptions, or immediately set a phone interview right then and there. After the two weeks are over you’re free to continue communication with each of the contacted companies until you’re no longer considered or until you finally get your dream job.

Things I liked

Loads of responses

The response rate is incredible. From my own personal searches, I’ve received less than 1/6th response per resume sent out. With Hired, I just create one profile and received 12 requests for interviews. That’s a 1200% return for 1 resume posted! If you’re looking just to see what’s out there, Hired will get you a solid returns on your time invested into the site.

Very Concentrated, Very Quick

I like how there’s a very defined start and end time on responses. It forces each company that’s gunning for your heart to get a fair start and lines up each company’s interview schedules together which allows you to have a more organized schedule. It also benefits you in that when multiple companies end up giving you job offers, you can use each offer as leverage to see which one loves you more.

Things I don’t like

You’re still going through the initial recruiting hoops

So this is what happens. Recruiters sit down Monday morning and browse Hired for the latest batch of candidates. They sift through and find some profiles where the skills and history seem to match the requirements of the job. They contact you to set up a call where they… have you… state your skills and history to see if you’re a good fit…

I don’t get it. The initial recruiting call seems highly unnecessary since a Hired profile gives way more information than a normal resume does. It also gives more information that a standard recruiting call will ever expose. I think this practice of screening the candidate is still done because internal company recruiting processes are slow to change and a concept like Hired which should make this step obsolete is too different from their traditional hiring process that they choose to leave this redundant step in.

I personally would rather talk directly to an engineer for the opening call but that was very rare.

Talent advocate sets up your one-liner description

Your talent advocate will size you up with a one-liner description after your thirty minute introductory call. This description is the first thing recruiters will see when they see your name. I actually wanted to look for more Node.js and fullstack JavaScript opportunities, but my talent advocate boiled me down to ‘Frontend developer previously with Experian; led multiple teams and projects to success’. I didn’t realize until after the two weeks were over that this was how I was being advertised which kind of explained why I got so many offers for front-end lead positions even though this was something I wanted to shift away from.

It appears now that Hired encourages you to contact your Talent Advocate if you’re not happy with your description. I’d like to think my complaining about this made a difference haha..

Most of the positions are going to be startups from SF

Home to all the hippest engineers, SF and Bay Area companies are begging for more talent. I originally wanted to look for positions all over the United States but my Advocate recommended I boil it down to three cities. I chose LA, SF, and Seattle. I got 1 response from LA, 1 response from Seattle, and 10 responses from SF. Out of those 12, 2 of those were large companies and the rest were Series A startups.

These kinds of results may change depending on the type of job you’re looking for but if you’re a front-end engineer or any web engineer, you’re going to be getting similar results.

All in all..

It’s a really good service. Its searching for jobs without actually putting in all the work of having to find them. Granted all the responses you’re going to get aren’t going to resonate with you and some companies that contact you sound like straight up scams but the quantity of offers definitely offset the few that aren’t quality.

Codewars

Codewars takes the concept of the ‘Code Kata’ and gamifies the exercises so programmers always have something to challenge themselves when they got a few minutes to spare. Challenges are usually short, ranging from minutes to a few hours. Most of the questions often have contexts associated with them so it always feels like you’re actually solving a potential real world problem rather using your tools as a programmer, even though the contexts are often fantastical in nature.

Things I like

There’s a clear progression that Codewars wants you to take. Challenges start from the fundamentals and rises slowly in difficulty so that you have a good basis of understanding before you move to the next level.

Solutions of all the other members are available to analyze after solving the problem. Finding out how other (more smarter) people solved the same exact problem allows you to gain insights into the gaps in your methodology. The solutions are bite sized so its easier to digest new programming features or new ways of organizing code that you’ve never considered before. I truly think that social programming is the way to get better at writing code and I think Codewars deserves a giant plus for this feature.

Things I don’t like

The Java error messages are less than helpful. I’m not sure how they return exceptions when your code doesn’t compile, but often times the console is going to return an exception from the enclosing outer stack that executes the code meaning you don’t get any useful stack debugging information, you just know somethings wrong.

Each question has tags associated with them describing what methods of programming you’ll use to solve the problem. These tags often give a hint on what to do to solve the problem which is kind of a problem since people look at these tags as extra meta information to give them a lead. I often like to come into a problem blind and find out myself what to do in order to get to a solution.

Overall

Overall I think it’s actually a really neat service. Like I said earlier, I’ve learned so much more about JavaScript than I ever would have if I were to just continue working on my own projects. I’m definitely going to use it again when I delve into Python.

LeetCode OJ

LeetCode OJ is all about the algorithms. There are no pretenses, you’re going to be solving very straightforward problems that have very defined lower bounds in terms of space and memory. These problems will often test cases on all edges of the spectrum and will only accept answers that meet the lowest bound limitation. Your preferred language is just a means to work around an algorithm. At the end of each problem, Leetcode ranks you up against your peers by how quickly your code executed.

Things I like

No muss no fuss, these questions ask you about algorithms and after enough exercises will hammer the common ways to solve them into your brain. I think this is a good second level after reading and solving the exercises to Crack the Coding Interview. There are tons more exercises here that touches on things that CTCI doesn’t which makes it a natural progression to the book.

The code editor is actually really good. Everything flows like a real professional code editor including quality of life improvements such as auto tab indentation, mass commenting, auto closing brackets, mass tab indentation, highlight bracketing, etc…

The questions are very direct. There are no fillers, no context. At first its a little ambiguous to know what values will be used to test your program, but after a while it becomes very obvious what kinds of edge case values Leetcode expects your program to process.

Things I don’t like

Since the online judge only wants answers that are strictly the lowest bound in terms of space and runtime, it’ll often fail passing solutions that are not the most optimal. It will fail an acceptable solution of O(nlogn) if the lowest bound possible is O(n). Sometimes you think of the correct algorithm, but implement it in such a way that it still exceeds the time limit. It is ambiguous to know what exactly is the bit of code to optimize in order to pass the test.

The community seems more concerned with writing concise code rather than best practice code. Discussion solutions are often compressed to the point where its difficult to read.

I really wish they would allow us to see each user’s submitted code so we can see exactly what we need to do to get the most optimization out of our language of choice. I did not feel like I learned anything about runtime optimization since my code always landed in the middle in terms of speed compared to my peers.

Overall

This site is great if you need some extra exercises to hone in on pure algorithm practice. I would not recommend it as a place to actually learn from 0% knowledge but more of a place if you’re looking for medium to advanced challenges. You have to be familiar with your language of choice as your language is just a tool you should be familiar with to solve a greater problem.

HackerRank

HackerRank is site that focuses more on the competitive nature of programming. HackerRank encourages you to participate in its many ongoing week long challenges. They support a ton of languages cover a wide range of practice problems ranging from algorithms, functional programming, linux shell cmds, and even AI.

Things I like

Since each challenge requests you parse a text file, it actually gives you plenty of exercises in reading in a file, parsing it, and outputting it. I feel like this is something that’s overlooked by a lot of challenge sites so its nice to see a site that allows you to practice that skill.

Each challenge has very clearly defined variables. Sometimes it’s actually kind of annoying, but each challenge will let you know exactly how the text file will be formatted and exactly what it will be testing on.

There are a ton of active competitions going on at a time meaning you can always test your chops against other programmers.

Things I don’t like

Its code editor is missing a lot of convenience features that the other code editors have. Sure it has code highlighting but other than that its very uncomfortable to use. The UI around the site is nice, but I feel like it has some work to do on its code editor.

Parsing a text file every time is kind of annoying. Some times you just want the values to be passed in as primitive data as if it would be used as a modular piece in a larger project. Since JavaScript doesn’t have a real file output stream (without node) it requests for you to use console.log() to output your answer, which is a small inconvenience.

Overall

I’ve used this the least amount so I’m not yet decided on whether I like it or not. The challenges are appropriately challenging but the text editor is not very fun to use. Its less of a site to practice but more to compete. Some of you may view this as the same thing.

Conclusions

Just about doing any of these is a good natural progression in terms of interview preparation and to generally be a better programmer. All three of these sites and many more like it definitely can only help since you’ll be exposed to a wider range of problems you never even knew existed. Of course, nothing truly beats the experience of working on a truly challenging real project but when those are few and far in between, then these online judges got you covered.

preface!
I have tried to remain objective in my learnings but there may be hints of bitterness here and there on the account of, you know, failing these technical interviews.

The only winning move is not to play

Interviewing is a losing game. All applicants are losing, its just a matter of losing the least. From the time you say ‘Hello’ you’re already losing. Your interviewers are expecting you to finish a task in a certain amount of time and once finished will compare your time and performance to that of your peers. They are not looking for things that will impress them but rather for things that will reduce their opinion of you. There is no way to ‘win’ an interview, only a way to lose the least. Say too many things that rub the interviewer the wrong way and you’re out of the race.

The odds are stacked against you

A typical technical interview path usually involves five or more gatekeepers you have to pass. You usually have to impress each of the following people in order to advance to the next level

1. non technical Company Recruiter
2. technical phone interviewer #1
3. technical phone interviewer #2
4. CTO/manager
5. Take home assignment
6. onsite interviewer #1
7. onsite interviewer #2
8. onsite interviewer #3
9. onsite interviewer #4
10. onsite interviewer #5
11. onsite interviewer #6
12. day two onsite interviewer #1
13. day two onsite interviewer #2
14. day two onsite interviewer #3
15. day two onsite interviewer #4
16. day two onsite interviewer #5
17. day two onsite interviewer #6
18. Congrats you got the job!

The amount of interviewers has been expanded to indicate a nightmare interviewing scenario. Your interviews will probably not look like this, but is not outside the realm of impossibility

Remember earlier when I said that technical interviews is a losing game? Imagine a scenario where you are actually a really good engineer (unlike me haha!), and you have a 99% chance of passing through each gatekeeper. This means that you ultimately have an 84% (.99¹⁷) chance of passing through each gatekeeper and landing that job. The odds are good, but not guaranteed. Now imagine if you’re a good engineer but you’re no Linus Torvalds and you have a 95% chance of making it through each phase. That means you ultimately have a 41% (.95¹⁷) chance of impressing everyone and getting the job of your dreams. If you’re not within the top echelons of programming you might as well start practicing burger flipping because the only thing you’ll be engineering is the perfect .99c cheeseburger at McDonalds. Okay, that was mean, but my point is, you can be very good at what you do and still be rejected because of an interview with a gatekeeper going very wrong, and that’s okay!

Fail Fast. Try, Try Again

Its okay to fail at these interviews. Failing a phone screen doesn’t say anything about your character or intelligence. Say you have a 40% chance of getting past all the gatekeepers and landing a job. The probability of you landing that job at the first company you interview at is 40%, 1 - P(successive failures). The probability of you landing that job after two seperate company interviews is at 64%. The probability of landing the job after three seperate company interviews is 79%! Okay lets say you’re a average engineer and you have an 80% chance of making it through each interview and there’s 8 interviews to get through. We first calculate the probability of you getting past each gatekeeper and landing the job (.80⁸) 16%. Ouch, not good. You have a 84% chance at failing to get a job at a company with 8 gatekeepers. Now we figure out if you only apply to companies that all use 8 gatekeepers in their employee process then you will have a 16% (1 - .84¹) chance getting in on your first try 30% (1 - .84²) chance on your second, 41% (1 - .84³) on your third , 51% (1 - .84⁴)on your fourth okay you get the picture. Apply more to different companies and the odds start turning in your favor. You will make it. The thing is, is that interviews are not all cut and dry like a math problem. There’s a lot of things you can’t control such as the amount of interview gatekeepers and your probability of passing through each interview. While we can’t ever get our acceptance probability to 100%, we can definitely do some things to improve those odds.

Do not be yourself

Its kind of psychopathic, but do not be yourself. Be prepared to take on a persona of the most attractice engineer that has ever graced this planet. Never willingly give away anything that might expose your humanity. Technical interviews are like a reverse Turing Test. Instead of tricking the user that you the are human, you have to trick the interviewer that you are in fact a robot. You do not make syntax errors, you do not need to consult apis, you do not need a calculator, you make informed decisions on every aspect of engineering. Working for a company is not an opportunity to research or make mistakes, companies don’t want to hear that, they don’t want to pay you for that. Its an opportunity for the company to buy you to offload work on the current team much like how they may buy an additional server for their load balancer. You are viewed as an additional core to their server cluster and you will act that way. You are not valued for your ability to learn quickly or your knowledge of all these cute esoteric language features. You are valued on how much raw computing you can do for the team. After you get the job feel free to be yourself and share all those cute little quirks about your favorite dead language, but until then, you must give off the persona that you are perfect.

Do not panic

We are not perfect. Nobody is. We are flawed as human beings. We are not a compiler. We are not an interpreter. We are not a server core. We do not know every computer science principle, algorithm, strategy, pattern, data struct, language api, language feature. There is so much we don’t know, and that’s okay. If you don’t know something and a gatekeeper deems you unworthy, so what. Who cares, you failed, move on. Its almost a guarantee that you will be asked something you don’t know and even after an hour of thinking about it and problem solving around it you still won’t know. It happens. The important thing is to not panic. You don’t think well, or even think at all when you’re panicking. Stay calm, stay cool, don’t look at the clock, don’t freak out. You will get questions you don’t know the answer to, you will feel like an idiot after hours of interviews, you will realize your ignorance. Its cool, it happens, just make sure you’re the calmest idiot in the room. Interviewers interview tons of people that don’t know the answer to their question. They ultimately want to see if you can figure it out in less than an hour but do so by being calm and collected. Don’t give the gatekeeper more ammunition to not let you move on to the next level by appearing hysterical.

Do not talk

Do not talk too much that is. Say you’re asked about a subject that you know truckloads about. Do not drone on and on and on hoping to impress your interviewer. They probably don’t care. Remember, they are only looking for things to dock points off of and talking too much is actually a negative. Give the general answer, then ask if you can go further in depth. Every single technical interview help book/guide recommends to keep talking during your interview because your interviewer doesn’t know if you’re thinking or if you’re just stuck. This doesn’t mean talk all the time. Its incredibly difficult to blurt out a stream of consciousness and think at the same time. Take the time to close the yap and think. Let your interviewer know you’re going to think for a little bit and actually do it.

And Lastly, Culture Fit

One of the most disparaging reads I’ve read was about an interviewer that was gunning for Google. If you haven’t read this blog, take a look at some of the articles. This person is dedicated, has a goal, and a company in mind, seems pretty intelligent and very motivated. Unfortunately about four months into the blog creation and after ~120 articles about programming and computer science concepts, he does not make it. Even after answering all the questions correctly he was not offered the position(granted, there is no way to validate this other than taking his word for it). Even if you are Steve Wozniak, Bill Gates, Linus Torvalds, Tim Berners-Lee, or Al Gore, sometimes they will reject you just because you didn’t gel with one of the gatekeepers. Maybe the gatekeeper was threatened by your intelligence, your smile, your dashing good looks, your charming voice, your chiseled chin, your flowing hair, whatever the case sometimes you get denied because you said, wrote, or did the wrong thing that may or may not be related to programming. It sucks, it happens, move on.

A way to get past this culture fit criteria is to be legitimately excited for the produce the company works on. Remind the interviewer before and after the interview about how excited you are about the product by asking questions about the product. How does it work, who builds it, who decides what goes in it? And if you aren’t actually interested in the product, then feign interest. ‘Wow creating A/B testing for ancient email web app to serve ads to a demographic that’s age 50+ sounds like the opportunity of a lifetime!’.

Conclusion

Be cool, remain positive, keep trying.

Prototypal inheritance is similar to classical language’s concept of extending. It has some of the same concepts of subclassing such as inheriting parent fields, overriding, and creating new fields. Here’s a small example of how to subclass a JavaScript class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

   //Parent class
   var Parent = function(){
        this.name = "parent"
        this.status = "adult";
        this.species = "human";
    }

    //Child class
    var Child = function(){
        this.name = "child";
        this.status = "infant";
    }

    //set Child's prototype to a new Parent object
    Child.prototype = new Parent();
    //re-set the Child's constructor to itself, since it was overwritten in the previous statement
    Child.prototype.constructor = Child;

    var child = new Child();
    console.log(child.name) //"child"
    console.log(child.status) //"infant"
    console.log(child.species) //"human" taken from Parent.species
    console.log(child instanceof Parent) //true
    delete child.name //true removed Child's name
    console.log(child.name) //"parent" taken from Parent.name

This is the basic structure. Each function has a prototype property. This prototype field is an object that is applied to all objects created from that function using the new keyword. Setting the prototype of the Child class to that of the Parent class will create a prototype chain which would then allow the instanceof feature to work. The Child class would also borrow any attributes it didn’t have that the Parent object would supply. Deleting overriden attributes in the Child class would revert back to use the assigned Parent’s attributes.

Prototypal inheritance in JavaScript is slightly more flexible than in a classical inheritance pattern due to the ability to extend Parent functionality without the Child knowing about it. Its not needed in a class signature but rather a simple assignment to Child’s prototype attribute. Its also possible to selectively choose which attributes to inherit from the parent. Not all public attributes need to be in the prototype object before assigning it to the Child.

Javascript inheritance is definitely not as straightforward as Java’s but there are ways to get the job done.

In case if you forgot…

• Data types are represented by bits
• 8 bits make a byte
• 1 byte (or 8 bits) make a boolean
• 2 bytes (or 16 bits) make a char
• 4 bytes (or 32 bits) make an int
• 8 bytes (or 64 bits) make a long [this is not shown in the chart]
• There are a total of 2^16 (~65k) character representation in Java
• There are a total of 2^32 (~4 billion) int representation in Java
• Each character and integer can be represented through hex
• \u0000, \uFFFF represent characters where each character represents a byte
• 0x00000000, 0xFFFFFFFF represent integers where each character represents a byte
• Each 2^(10*n) breaks another thousand barrier

So say you have N amount of bytes to work with. If we want to know how many ints we can fit in there, we take N/4 since each integer uses four bytes.

Say we want to fit all integers into an array in memory. How much memory to do we need? we take all the integers (2^32) then multiply it by 4 since each integer is represented by four bytes. We would need roughly 16GBytes of memory to put all representable integers into an array

The conversions are for the most part simple, it just requires careful calculation around big numbers.

website
source code

6.82 Visualizer is a gold and experience data visualizer (grapher) for the game Dota 2. There was a controversial update to the game where the game would give the losing team drastic advantages so that they would be able to catch up to the leader. The changes are difficult to imagine due to the amount of variables so this program was created to show the data on how much the game had changed.

For those unfamiliar to the game of Dota, there are two teams of five that duke it out against each other to ultimately destroy their opponents base. Just like a real war, the way for a team to gain an advantage is to have more money, that’s essentially what Dota’s gold and experience is. Many members in the community were worried that the change to give the poorer team a boost to their economy would change their game for the worse. Many who were vocal about this had personal anecdotes but very few had the hard data. I sought to provide the community with the hard facts in an easy to view format of how the experience and gold were exactly affected so that we could see clearly just how much it altered the game.

The Backend

Backend? What backend? There actually is no backend to this site. The entire application is self-contained in the html and js file. The page is served through GitHub’s project pages much like this blog you’re reading.

The Frontend

The front end was built using a charting library chart.js as well as angular.js for the data binding. In the angular file, I have one controller for the form that maintains all the adjustable data. I knew that angular would be a good fit for this project since it very easily binds changeable data to the view. I also created a ‘chart’ directive attribute tag. This chart directive contains code to initialize whatever element it was inserted into to become a chart. And finally I had one factory that contained all the formulas for calculating the data. I tried using as much as angular offered in intelligently splitting up the tasks of the site.

The good, the bad

This is actually the first site I made that was incredibly well received. Unlike my other sites, marketing this one was no problem. I posted it once on reddit/r/dota2 where it quickly rose through the ranks with nothing but positive praise. I felt very alive after receiving the first bits of feedback. I quickly continued to work after the initial release creating more graphs, more analysis, preset configurations, sharable configurations, layout changes (you can actually see the history here). I keep realizing that community feedback really drives me forward and motivates me to a degree that I can’t find anywhere else. Community figures were tweeting about my site, forums and fan pages in China, Germany and Russia were referring to my page, the page was being shared organically through Twitter and Facebook. It was actually quite amazing for me. I’m really proud of the fact that I was finally first to market in something. Usually most of my ideas are imitations or alterations on an already existing idea, but this time I got lucky.

That’s all well and good but of course there’s some bad. The bad is that the page is not optimized in a degree that I would be proud of. Granted, the application is very JavaScript heavy in the amount of calculation and redraws required from both the charting library and angular but playing with the sliders produces choppy animation. Optimizing that page to for its redraws to be at 60 frames per second was not my highest priority but it would be a good assignment to bury myself in (later). I’m also a little ashamed that I wasn’t able to produce a proper mobile/tablet view. According to Google Analytics, about 10-15% of hits were coming in from tablets which is expected. I anticipated a mobile design view would have taken too much effort for it to be worth it. I could only have imagined a site redesign with unique scripts to support a program with this many charts, knobs, and buttons.

Well there you have it. Although this site might not be useful to you random reader but it definitely should be useful to the Dota 2 enthusiast.

website
source code

Share Codes is a site for people to trade beta/alpha/early access codes. The premise is simple, people with codes post their email and people who want codes post their emails. Match made in heaven!

This site was originally created for people to share the early access demo code for the video game Super Smash Brothers for 3DS. Only a few privileged people were able to receive an early access code from Nintendo and had a few codes to share. Internet communities across the web were going wild on searching for a spare code while those fortunate ones with the codes had the opportunity to get something in return for having such a rare commodity. I felt a site like this was needed to help people filter all the noise into something more digestible. The site was furiously written and released and it even helped some people find and give away codes.

The Backend

The backend was made with node with the express framework. Once again I tried to make a proper REST API style endpoints that my front end would connect to. All emails are stored in mongodb through mongoose. The server has a scheduled timed function that prunes emails from the email list if the email is older than 30 minutes. That’s it, there’s nothing more interesting to say about the backend.

The Frontend

The front end was created with plain old html with the bootstrap framework for styling. I don’t really like using the bootstrap grid layout feature so I don’t use that and make my own responsive layout using flexible containers and media queries. The form posting of the emails is not actually asynchronous. I know the experience would have probably been better if emails asynchronously updated without a page refresh but I haven’t worked on a non-asynchronous site in so long that I really wanted to remind myself how to do it using traditional forms, actions, and methods. Its funny, I’m a developer born into the web 2.0 world that I have little project experience on how to do things in the web 1.0 world. I also wanted users to keep refreshing their pages to simulate heavy load to test how much strain a single Heroku dyno can take.

The good, the bad

The micro app was a blast to make. I knew that there was a time limit of relevancy since the codes would expire in a week. I churned this site out in around six hours just to see if I could. I think I felt that this would be kind of a proving ground whether I’m comfortable enough with node to create a basic web application relatively quickly. Fortunately I was able to reuse the framework from my other node projects that already had the configurations I wanted. The site worked and I felt really alive after I saw actual users on the site. I continued to make small improvements to the layout, adding extra fields, cleaning up text, making it responsive, all within the first hour of being live. If I wanted to generalize this, I can just swap out the title and make it apply for any kind of pre-access code sharing site.

So here’s the bad. The bad involved the marketing. Once I was done, I sent it around to my friends, then I made a post on Reddit in the primary thread and any other thread involved in trading of codes. Unfortunately for me, these threads were already filled to the brim with people looking for codes. It was a storm of fish desperately trying to gobble the bait. I think it was the largest Reddit thread I’ve seen at 20,000 comments since I last seen it. Every time I would make a post in a thread I would check the analytics and I was fortunate to even get a few bites. Someone even posted it on their Facebook wall thereby getting a few more bites. It was pretty swell, my site was actually being used by real people. Many of them posted their email looking for codes but nobody posted their email giving away their code. It was a seller’s market at a rate of 1 seller to 50 buyers. Some users kept reposting their email when it got pruned but then eventually the site would be empty.

Well, there you have it. I’m glad that some people were helped by the site, but it didn’t spread even when I hopped around forums posting the link. I think the clarity of the share codes service is actually really helpful but when people are already ingrained in their communities it seems that they can’t be bothered to use anything else.

website
source code

Quiz With Me is a community generated trivia site. Each user competes against the people around the world to test themselves with the trivia questions that they submit. Quiz With Me brings the joy of answering and creating trivia questions vs the entire world!

Every player on Quiz With Me are all competing against each other directly in the same trivia game. Each player gets time to answer the same question as everyone else. Once time is up, the polls are tallied and users can compare their choice to those of their peers. Players can play anonymously or sign-up to keep track of their points. Each player can also submit their own multiple choice trivia question with at least four or more potential answers. Each question can also be judged by the community as to whether it is factually correct or just nonsensical BS.

The Backend

The backend is ran on node.js with the express framework with a mongo+mongoose persistence layer. This is my second project working with node and I wanted to follow REST API guidelines when creating endpoints. You would think I would use socket.io again to create real-time experience but I really wanted to practice working on creating a RESTful API. Its funny, all API creation guides recommend making your server server-side content stateless. The problem here is that I needed to synchronize all users to the server’s state, the time remaining on the current question, and the question itself. I think its easy to create a stateless REST endpoints when dealing with normal documents, but when we’re creating a game of sorts, its kind of assumed that we’re going to be maintaining a state that all clients have to synchronize to right?

The Frontend

The front end is composed of a single index.html page that contains a bootstrapped themed skeleton and an angular.js app file that fills it all in. The angular.js file has all the routes inside providers that asynchronously contact the server endpoints to get new questions, or handle logins and registrations, or create new questions. There’s controllers for the questions segment, a controller for the poll segment, and controller for the header bar. Each of these controllers does what angular does best which is watch for changes to objects inside the controller which automatically update the tags in the front end. The page is also responsive in that the buttons and content will reorganize themselves based off the window size. There’s slight display issues regarding the header bar in the mobile view but I’m generally happy with how I finally got to try out this new framework everyone loves to hate.

The good, the bad

Of course, competing with your friends and strangers across the world would be the most ideal scenario but the problem is the lack of a user base. The site functions well with all the features of account management, question submission, question tribunal, in an acceptable presentation but all of this doesn’t really matter unless someone is using it. The design is also very bare. At this point I’m happy that the functionality works as making the front end look polished was never the goal. The site suffers from terrible SEO due to the angular generated front end that dynamically fills in content. I tried to remedy this by adding an ‘about’ light box describing the site but it doesn’t seem to help. I’m still unsure how to organically get hits for web applications like these without actually doing the work of marketing it first. Its making me believe that the new Google SEO algorithm is dependent heavily on public impression and how well it does on social network sites, Twitter, Facebook, Reddit and so on. The goal of this project was to continue working with node web applications and to get my feet wet with angular. With the completion of this site, I feel like I graduated from the kiddy pool and ended up in the deep-end.

website
source code

Confess.me is an anonymous real-time secret posting website. Other supportive members can send comforting messages of support to the anonymous secret sharer. The secret sharer can thank these anonymous eyes and ears for their support in turn. All of this is done in real time letting users have an outlet for those things that they hide to themselves.

The Backend

This site was made with node with express with the addition of the socket.io plugin for websockets. One of the tricky things to manage was sessions. Websockets aren’t actually remembered within the same session so it was necessary to create a session to websocket object storage in the server memory. This ensures that users maintain the same websocket identity as long as they keep the same session identity. After that it was smooth sailing. The socket.io documentation is very basic but explains all there is to know about real-time communication. Set the server up to listen for events, have the client send those events. Have the client listen for events, make the server send those events. Its very basic but I can imagine so much more could be done with sockets ranging from real-time games, polls, data, and even parallel computing.

In terms of persistence, there wasn’t supposed to be any message persistence. Lacking message persistence gives each message a fleeting property as if your secret will be seen but soon let go and forgotten. This was the initial plan, but since there is a severe lack of a userbase many new users would be confused and see nothing on the homepage. I felt the need to store some of the latest messages so that new visitors will see that there’s actually some content. I use the hip new document storage system mongo wrapped with the mongoose framework. The mongoose framework includes the ability to create object schemas for the mongo database. Even though mongo is naturally a schemaless storage system, I feel like it gives too much freedom for the user which would ultimately result in errors and confusion. Mongo as itself is a pretty simple concept to understand for people already familiar with JavaScript objects. My only wish is that it would carry some familiar SQL syntax for table searching but unfortunately its a new syntax, still simple, but new. Mongoose required a bit more reading to understand though. Its documentation is decent but I never really liked frameworks.

The Frontend

The front end is pretty simple, uses the client side javascript for socket.io and some extra javascript to accomodate the multi-column layout. The front end is media query responsive so it’ll shrink itself to as many or little columns needed on smaller devices. It was fun trying to get the messages to organize themselves based on a multi-column layout vs a single column mobile layout. My rule for the multi-column layout was to always places the message in the smallest column as well as have newer messages up top, that way all column heights would be relatively even. When users resize their window, messages get reorganized so that it always follows this rule. I also used google’s third party library on html sanitization to prevent the page from accepting links and to protect users against XSS attacks. The problem with this real time updating of content is that the site has zero SEO. All content is dynamically injected through javascript so there is no content to scrape on the index.html page. To be fair though, I didn’t really optimize the page to be picked up by search engines, otherwise I would have included an about page, or help, or a blog, or something with actual content, or server side rendering, but at this moment I’m just happy that its primary function works, even when there’s no audience.

I’ve always been fascinated with real time communication and what that could mean for the web. I thought for this project having someone respond with immediate results would be better than having to refresh the page constantly to see if someone replied to your secret.

The good, the bad

The good part about the site is that it works. The bad part about the site is that it has no user base so it can’t really function as intended. Ideally there will always be more than a single person on, but at this point it has no user base other than my multiple testing browser windows. At this point I’m not sure if it can even handle more than five people at a time much less a hundred or even a thousand. Currently the code is sitting on a free Heroku dyno, I never intend on taking it to a real production level plan unless it happens to explode with popularity which I doubt. This was also my first experience in javascript server side programming. It was interesting imagining how code jumps around to different functions in a non-sequential manner and to be honest, I’m still not entirely sure how to handle errors. I haven’t yet fallen into callback hell but I can imagine that it can get pretty nasty in larger applications.

Well there you have it. I hope this site will be useful as an outlet for the things that trouble you.

Source code

NicoNicoTwitch is a JavaScript canvas project that’s meant to bring over the unique on-screen scrolling commentary feature of Nico Nico Douga onto the popular game live streaming platform twitch.tv.

I made this because I always loved Nico’s unique text-on-video overlay of community comments and wanted to bring that over to Twitch who also sports a vibrant community. I personally think that bringing the chat closer to the video gives us a closer connection to those on the people on the web as you can relate when you both find a certain scene uproariously hilarious, or another particular scene sad. Twitch was the perfect video playing platform to emulate this feature on due to the chat being in real-time as well as being quite infectious with its commentary.

The basic rundown of how NicoNicoTwitch works is pretty straightforward. Set a timer to read any updates in the chat, when there’s new messages, parse and process them for any emoticons, send them to the display sorter where it’ll find a spot for them to ‘marquee’, then update and animate the text. The jQuery library was used for its convenience in dom selection and traversal through all the chat elements and the HTML5 canvas feature was used to paint the text over the video. The following will go into a bit further detail about the intricacies of the program.

Running it

So how do you run it? Well, at this point there is no proper executable. Its simply a hack that’s supposed to be ran through the browser JavaScript console. Just go to a twitch streamer page with chat (i.e. not the homepage) open the console (f12) and paste the code in. I’m not exactly sure how to package this up if I want to release it publically. I hear Chrome Apps is pretty easy to use in terms of packaging up JavaScript applications but to be honest, sharing this wasn’t my priority, it was mainly done for the practice.

The Nitty Gritty

So what happens when you paste the code in your console? First, I use jQuery to identify where the video flash element is on the screen and overlay a new Canvas element directly over it. This canvas element is transparent and will be used for our flying text. I insert the ‘Initialized’ text message into the message processing queue where the message is then processed and inserted into a TextObject where it is given its current x and y coordinates as well as the width of the message, the speed at which it’ll be flying across the screen, the color of its user, and of course the text message itself.

Inserting the Text

The insertText function decides the initial location of the text on the screen. New messages captured from the dom get passed into this function. I gave two rules to deciding the text location.

1. 1. new text shall be placed as close to the top as possible
2. 2. text should never overlap each other.

Ensuring that text wouldn’t overlap each other essentially required that any incoming message wouldn’t start to leave before the already outgoing message left the screen. This required a bit of math involving the velocity of text, the frame rate, the canvas width, the width of the text itself, and the width of any emoticons inside it. I was very fortunate that canvas already supplies a method that measures the width of any supplied text with context.measureText(“text”). This took out the grunt work of measuring each pixel width of each letter individually and calculating the width of text messages letter by letter. I was unsure of whether my insertText function was going to work until I implemented my animator.

Updating the Screen

My animate function is pretty straight forward. Clear the canvas screen, update the location of each active text objects based on their given velocity, draw the text back on the screen. When all is over, it calls a global function requestAnimFrame that calculates when to call the animate function again. Currently the Frames Per Second (FPS) is set at 30, so each animation frame triggers every 33.3ms. The FPS is completely adjustable through a variable but I think it looks best at 30. If I didn’t time the animation frames to run in a constant manner, text would probably speed by unnaturally fast as the program would be running the animate function as fast as it could.

Parsing the DOM + Messages

Okay! We got our animate and we have our insertText function, now we just need to read the messages from the chat box. There’s another timer that triggers every second to read messages from the chat box. It only reads up to the latest message that it hasn’t seen before and sends it to the insertText function. This is entirely done with jQuery doing the searching in the DOM tree. What was probably the most challenging here was parsing the html to grab only the information I wanted with regular expressions. I’m not particular with regular expressions but it was good to practice my matching abilities. What was most surprising to me though was that JavaScript’s RegExp.prototype.exec() only returns the first match. If there are multiple matches in the text for the same regular expression, you need to keep going in a while loop until the exec() returns a null. Why was it designed this way? I think that’s very counter intuitive. The reason why I needed to exec multiple times on the same string was to catch multiple uses of emoticons in the same message which as we know Twitch chat loves spamming which makes this a very valid use case.

Parsing the Emoticons

Of course getting emoticons to appear on the canvas object was not as easy as I thought. I needed to parse out the emoticon html, identify the graphic url, identify the graphic keyword, and have the animate function insert a picture in between the text messages. Before the recent update, Twitch used to display their emoticons using <span> tags that had background-images in CSS. Since Twitch has an ever updating emoticon list, I had to preload all possible emoticons and identify each emoticon’s url by going through each loaded span and identifying its background-image. Of course this wasn’t really that great due to having to pre-load more than 10,000 emoticons before running the program. With the recent emoticon update (which according to the blog updated 4 days ago) Twitch made it way easier to identify the url for each emoticon since they now use an actual <img> tag that has the src right inside it. Now I use a system that lazily loads the needed emoticon as soon as someone uses it in chat.

The Annoyances

Of course with such a successful project there are always some things I’d like to improve on. It can always be more polished obviously but I’m happy at the point that it’s at now. I can put further filters in text parsing to remove anchor tags. I can create an actual emoticon object that keeps track of each unique emoticon’s width since as of now I give each emoticon around 40 pixels of space. I can refactor the code just sitting out in the open in the NicoNicoTwitch function into its respective config, init, or manager classes instead of barfing everything in the same js file (of course this was before I knew about require.js). I can also clean up comments. As of writing this :) and <3 are not processed correctly by the emoticon parsing system. I also think there’s a better way to read new messages from the chat box than to check it every second.

But with all these annoyances that are actually under my control, the one true annoyance is with Twitch itself. It’s not that I can blame them, but every time they update their system to use a different identifier, different dom structure, different emoticon keyword, different emoticon url, my program breaks. The fact that my program is completely dependent on them not changing anything to their site is the greatest annoyance. Of course, this was before I knew about their public API that exposes each channel’s chat. Regardless, I really can’t complain since it is expected that Twitch works on their site to make it better.

Well that was a long boring read but actually watching it come together is something like magic. Its fun seeing two messages on the same line coming close to colliding but actually just barely miss each other. It’s also fun when there’s a lot of chatters filling up the entire screen cheering with excitement or sharing sympathy with a solitary BibleThump.

Sections

1. Finding the IP with DNS
2. Creating a connection with TCP
3. Requesting and Sending data with HTTP
4. Rendering the DOM

Finding the IP address with DNS

So the first thing your browser needs is a website hyperlink so it can ask it to bring back what it finds there. Okay perfect, you got one, youtube.com, google.com, facebook.com, all your favorite sites all have names. Unfortunately, computers don’t handle people names very well, what they need is an IP address, a series of four separate numbers from [0-255] divided by periods. Fortunately for the browser there is a distributed library of domain names to IP address mappings that the browser can use to find out the IP address behind the domain. This ‘library’ is called the Domain Name Server (DNS).

If I may extend that terrible library analogy, think of the configured DNS as a librarian and all the domains and IP addresses in the internet as the books. There are many Domain Name Servers in the library of IP addresses just like there are many librarians in a library. When you ask your local librarian to find a book for you, they may know where it is immediately, or they may need to ask someone higher. Same with DNS’s. This someone higher is mainly just a manager of all the other librarians that fall under their specific domain and will redirect you to a higher librarian that might have the knowledge. This can repeat several times up the chain as there are a lot of higher level librarians to consult. Each higher level librarian may have the answer, but if they don’t, then the query goes all the way up to the Head Librarian. The same applies for DNS’s. Each DNS might not have the answer but each has a list of several others it can consult to find the answer. This asking around continues until it reaches the Root Domain. Root Domain’s don’t have the answer directly, but they know the perfect librarian who does. The way that Root Domains know which DNS to use is by checking the domain of the url, or the rightmost part of the url. If you’re asking for a url that ends in .com the root domain give you a name server that specializes in domains that end in .com, if you’re asking for a url that ends in .net the root domain will give you a name server that specializes in domains that end in .net. Your original librarian or DNS will finally get their answer and pass it back to the requestor, or the browser. You can query the DNS yourself to find out the IP address of your favorite sites with a tool like who.is. Try putting your favorite site in there and copying and pasting the IP address in your url. It may seem strange to put a bunch of cryptic numbers in your browser but that’s essentially what’s happening in the background whenever you type in your favorite site.

Creating a connection with TCP

So now that the browser has the IP address we just need to send a message asking the server that’s hosting the website to give us the webpage, but before we do that, we need to check if anybody is even there to respond.

If we may continue using terrible analogies imagine if your browser and the server are testing a snail-mail delivery system for the first time and they want to validate that they are able to successfully send each other letters accurately. Your browser initiates the conversation by sending a letter to the server’s IP address that has a header saying “Hello, my name is ‘Browser Client’. My sequence number is 10000. Please acknowledge back with my sequence number + 1, and your sequence number so that I know you got this message accurately”. Days pass after sending the letter until finally the mailman comes with a response from the server. Browser hurriedly scans the headers of the letter which reads “Hello ‘Browser Client’, my name is ‘Server’. I acknowledge your sequence number is now 10001, my sequence number is 90000. Please respond back with my sequence number + 1 to confirm if you got my message clearly”. Finally the Browser Client sends one final handshake to seal the deal and open a connection, this message contains something like “Hello ‘Server’ nice to hear from you. I acknowledge that your sequence number is now 90001”. The acknowledgements that each client and server send to each other let them know that their message has been received and that they are now synched to each other and ready to receive more data. So as you might have guessed, the letters represent TCP packets, the header of the letters represent the packet flags, there really is no body in the letters yet since we’re just trying to establish a connection and have not yet passed data (other than the information provided in the packet headers), and the postal service are the routers and switches that deliver the letters. A successful TCP connection allows both Browser Client and Server to start sending each other data.

An open connection means both client and server can start sending each other text, images, videos, binary files, anything as long as it can be serialized into bits. Sequence numbers are randomly generated by both sides and the amount of bytes that are passed are remembered and added to the initial sequence numbers. This ensures that each side has a way to ensure that the proper amount of bytes have been received by checking the updated sequence numbers from the receiver being equal to the amount of bytes sent in the previous transmission. This is why TCP is considered to be a reliable protocol because it will catch and fix broken packets containing incorrect messages.

Requesting and Sending Data with HTTP

So now we know the address, and now we’ve secured a connection, its time to start checking what our friends are doing on Facebook or check out that new movie on Netflix. The way that information is passed between the client and the server is through the HyperText Transfer Protocol (HTTP). The protocol kind of defines the language that both client and server will be using when communicating with each other. There’s a set amount of agreed upon actions that can be used when communicating through HTTP. The most commonly used words used are GET and POST. GET usually indicates the client receiving a file from the server, its akin to a READ action in terms of local storage. POST usually indicates a transference of data that will change something on the server, akin to a WRITE action. Lets simulate a GET request.

The Browser Client sends a GET request asking for some content lets say an index.html file. This GET request contains information that the server program would use to process and send back the appropriate file/text/image/binary file. The server software can read extra parameters like the URL, the attached cookies, and the user-agent to decide which kind of response to send back. The server then responds back with a response packet usually with a data payload along with important headers like the status code, content-type, and content-length so that whoever is receiving it can interpret the data correctly. In this case, the browser sends the GET request for index.html to the server and waits for a response. The server receives the GET request for index.html and sends back a response packet with the index.html in the body and sets the status to ‘200 OK’ and the content-type to text-html. The browser then receives the response packet, identifies that it is an html file and displays it to the user appropriately. If the content type is an image, it’ll try to render an image from the data, if the content-type is JSON it’ll render try to keep it in JavaScript Object form, if the content-type is XML it’ll try to display the XML, but usually, the content type is plain old html.

POST requests are kind of sent in the same way. The primary difference between a POST request and a GET request is that POSTs are not restricted in the type and size of data they can send. As I said earlier POSTs are usually used to indicate to the server to change or update something. Usually the role of POST gets a little muddled with other HTTP request methods such as PUT and DELETE which places and deletes the requested URI resource respectively. This has kind of been a problem with the web as these HTTP actions are not hard rules but rather guidelines open to interpretation. Most of the action verbs do the same thing, carry a URI with the same headers with an attached data body. This is why it’s up to the developer of the server software to decide how to handle the action verbs. Most of the time it just boils down to, if it changes something on the server, make it a POST.

Why do we use HTTP and not a protocol that’s more suited toward the modern web? Why do we still use a protocol that can only send one file per request and have to open a new connection with the same three-way handshake for each request? From what I’ve read, it’s because of legacy reasons and pretty much the fact that the web grew up with the protocol. People are just used to it now and are slow to change. Firewalls have been defaulted to accept requests from the standard http port 80 which allows communication to occur between server and browser. There has been a push for an upgrade to the http protocol called HTTP/2 which is working on reducing the constant request overhead by compressing header information, minimizing constant handshaking, stuffing all needed data into a single TCP connection, proactive server responses, amongst other cool things that’s certain to speed up the web even more. The great thing is that modern browsers and web companies are actively trying to support this new protocol in the race to be the fastest browser as well as be able to serve their content as quick as possible.

Rendering the DOM

So we have the address, we have the connection, we sent the request and now we have our index.html file, now all the browser needs to do is show it to you. Since the content-type is an html file, the browser will do what it does best and that’s create the Document Object Model (DOM). The DOM is a constructed object tree created from the nodes in the html file that’s used by the browser to more easily process the current html file. While the browser parses through the html file and creates the DOM it also creates new http GET requests for each extra resource it comes upon such as CSS files, js files, images, tracking tags, Iframed content, and so on. The browser grabs all available CSS files and generates a corresponding CSS Object Model (CSSOM). The CSSOM is applied on the DOM which ultimately creates the Render Tree. The Render Tree notes the dimensions, layout, and location of each document object awaits the draw phase. If we want to continue with the analogies, I guess we can compare the Render Tree to say the final sketch of painting before we color it in permanently on the canvas that is our browser window. The end result is your fully painted index.html homepage.

Of course this how it might have been fifteen years ago on very generic simple pages, but today we’re using a ton of cool technologies that calculate and create custom generated Facebook feeds, selective YouTube recommendations, Netflix movie suggestions, tailored Amazon shopping experiences, all within milliseconds. Today, html pages are rarely ever static, but are influenced by a wide range of things from the movies that you watch to the things that you buy to the sites that you visit. All of this advanced and cool things are still possible just by the simple bits of data that’s passed along with the very basic HTTP requests. We haven’t even got to the ability for sites to dynamically update with the inclusion of JavaScript but you can imagine how that adds another level of interactivity to a web page.

And That’s That

This explanation really is just the start of it. There’s so much more that goes down in each phase that its pretty incredible how something like this can even come together at a speed that’s actually somewhat fast. The arms race for browsers to render even faster and companies to serve their content as fast as possible is well underway and is definitely going to be the focal point of web development in the coming years. Its amazing that even with such a simple communication protocol we’re able to get the web to where it is today, as a marketplace, as entertainment, as a collaborative platform… Communication at the speed of light sure can achieve great things.

Resources

More information about DNS
More information about the Three Way Handshake
More information about Request and Response headers
More information on how browser rendering work
A more simpler version on how browser rendering works

If you come primarily from a classical object-oriented programming background, ignore everything you already know about this. The reference to this is very dynamic. this can reference any object depending on whether we specify the reference explicitly in the code, or whether we leave it up to the interpreter to assume what we want this to reference. Most of the times we choose not to explicitly set this leaving the reference to be decided by the interpreter which is where most of our issues and confusion comes in. While you can memorize every instance in how this is going to be resolved by the interpreter, there are only a few context and concepts to know about this where you can come to a conclusion yourself in any situation. Here’s the main contexts where you’ll be seeing this used in and how you can make it work predictably for you every time.

this in a constructor

Here’s what happens when you use the new keyword to create a new object in JavaScript. The following text is an altered and truncated version from the Mozilla Developer Network on what happens when creating objects with the new keyword.

1. A new object is created, inheriting from Object.prototype.
2. The constructor function Object is called with the specified arguments and this bound to the newly created object…

The MDN docs uses an interesting word, ‘bound’. From this point on, whenever you see the word bound or bind, you should associate it with setting the this reference. If I say functionA is bound to ObjectA then you should think that whenever functionA uses this it should always reference ObjectA. The MDN documentation says that it first creates an object using Object.create(), this step is not really that interesting as it relates to this, it’s just merely setting up a shell of an object for us to bind to. Then it executes the constructor function where we start assigning properties to this, or rather the new object shell we just created in step 1. What this means is that when the function is executed, the interpreter starts attaching properties to the shell object. The example below codes out what the MDN doc is saying

1
2
3
4
5
6
7

    var Person = function(){
        this.age = 20;
        this.name = 'Jason';
    }
    var person = new Person();
    console.log(person.age) //20
    console.log(person.name) //'Jason'

The above code is equivalent to the code below

1
2
3
4
5
6
7
8
9

    var Person = function(){
        this.age = 20;
        this.name = 'Jason';
    }
    var person = Object.create(Person.prototype);
    //this is where the binding happens 
    Person.call(person);
    console.log(person.age) //20
    console.log(person.name) //'Jason')

Note in the code above this line Person.call(person). What that line means is that the Person function is executing and explicitly making the empty person object equal to this. This means that after that function call, the once empty person object now has two properties that were applied to it, person.age and person.name. The following bit of code is identical to what the line Person.call(person) is doing above without the use of this.

1
2
3
4
5
6
7
8

    var person = {}
    var Person = function(){
        person.age = 20;
        person.name = 'Jason';
    }
    Person();
    console.log(person.age) //20
    console.log(person.name) //'Jason')

If you need more information about the function method call consult the documentation.

You may be asking, what if the function was just executed without specifically binding the this reference? Well that’s what we will find out in the next topic

this without binding

So what if we execute a function with this without explicitly defining a binding? Then we leave it up to the mercy of the interpreter. Most of the time, this will be refered to the global object. The global object varies between JavaScript interpreters. In the browser the global object will refer to the window object, in Node.js it’ll refer to global object. In either case, unknowingly attaching objects to the global object is probably not what you wanted to do.

1
2
3
4
5
6
7

    var unboundFunction = function(){
        this.param1 = 1;
        this.param2 = 2;
    }
    unboundFunction();
    console.log(window.param1) //1
    console.log(window.param2) //2

Note how when the function was executed without a binding, it automatically defaulted to the global object which in this case is window. There are many situations in where you’ll be wondering why your this reference didn’t work, usually its because you didn’t specify a binding to this and made the interpreter guess. While this is the main problems that arises from the improper uses of this the following can give you a head start on some of the more common tricky uses of this.

Bind when passing around functions

A good rule of thumb is to always bind objects to functions when passing them around as parameters or when returning them. Often times functions are passed around and users are shocked to see that their this doesn’t actually reference their expected object. To prevent this from happening to you, use the function method bind.

The following example passes a function to an Executor function that runs any function that gets passed into it. We create a person object and pass in its growUp function to the Executor where we think it will update our person’s age.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

    //a function that executes any function passed into it
    var Callbacker = function(callback){
        //do some logic
        console.log('executing callback function');
        //then execute callback function
        callback();
    }
    //creation of a person object with a growUp method that ages the person object by 1.
    var Person = function(){
        this.age = 10;
        this.growUp = function(){
            this.age += 1;
        }
    }
    var person = new Person();

    //confirm the property window.age to see that its undefined before method execution
    console.log(window.age) //undefined

    Callbacker(person.growUp); //'executing function'

    console.log(person.age) //10(
    console.log(window.age) //NaN

As you can see from the example above, passing the person’s growUp function to be executed by the Executor causes the function to bind the global object to this by default. That is why person.age doesn’t change, and window.age went from undefined to NaN (since undefined + 1 = NaN). If we explicitly define the binding before passing the function, then we can ensure that the growUp function will use the person object no matter where it’s executed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

    //a function that executes any function passed into it
    var Callbacker = function(callback){
        //do some logic
        console.log('executing callback function');
        //then execute callback function
        callback();
    }
    //creation of a person object with a growUp method that ages the person object by 1.
    //note the .bind at the end of the growUp function
    var Person = function(){
        this.age = 10;
        this.growUp = function(){
            this.age += 1;
        }.bind(this);
    }
    var person = new Person();

    //confirm the property window.age to see that its undefined before method execution
    console.log(window.age) //undefined

    Callbacker(person.growUp);

    console.log(person.age) //11
    console.log(window.age) //undefined

Note how the age now properly adds on the person object and how the window object is not affected. This is because we use the bind method that every function has. This specifies which object to use to reference this in the function.
This example is a little contrived but it’s definitely important to keep in mind especially when dealing with callback code or asynchronous JavaScript where functions are passed around like candy and the flow and scenario is exactly like that in the examples above.

Careful about functions that return anonymous functions

Be careful about your this reference when dealing with functions within functions. The following example is super contrived but here’s the rundown. Imagine a person object that’s at a holiday party and has a property for which drink he’s having. A person that’s under 21 will be drinking eggnog while a person 21 or over would be drinking booze. The person has a drinkFunction that will return him another function that updates the state of his drink to either ‘eggnog’ or ‘booze’. (I understand that this can be done with a single function but please bear with me.)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

    var Person = function(){
        this.age = 10;
        this.drinking = '';
        //top level functions like these are autobinded by default to whatever they're attached to.  In this case, the same 'this' object that the constructor binds to.
        this.drinkFunction = function(){
            //'this.age' resolves correctly and pulls the age property of the current object due to the function properly binding to the corresponding person object.
            if(this.age < 21){
                //this function is not bound to anything. It defaults to binding 'this' to the global object
                return function(){
                    this.drinking = 'eggnog';
                }
            }
            else{
                //this function is not bound to anything. It defaults to binding 'this' to the global object
                return function(){
                    this.drinking = 'booze';
                }
            }
        }
    }
    var person = new Person();

    //confirm the property window.drinking to see that its undefined before method execution
    console.log(window.drinking) //undefined
    //execute the drinkFunction as well as the function that it returns
    person.drinkFunction()();

    console.log(person.drinking) //''
    console.log(window.drinking) //'eggnog'

Note how once again, the person.drinking object wasn’t the one that was affected, but rather the one in the global window. This is because we executed the function that we returned in the global state meaning the this.drinking property in the inner returned function was by default bound to the window object when we executed it. If we want the this object to reference the same person object that’s executing it, we either need to bind the internal returned function or call the function when executing. Take a look how at the example below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

    var Person = function(){
        this.age = 10;
        this.drinking = '';
        this.drinkFunction = function(){
            if(this.age < 21){
                //note the binds to the same top level 'this' object.  This ensures that the anonymous function references the same 'this' object that gets created when new Person is constructed
                return function(){
                    this.drinking = 'eggnog';
                }.bind(this)
            }
            else{
                //note the binds to the same top level 'this' object.  This ensures that the anonymous function references the same 'this' object that gets created when new Person is constructed
                return function(){
                    this.drinking = 'booze';
                }.bind(this)
            }
        }
    }
    var person = new Person();
    //execute the drinkFunction as well as the function that it returns
    person.drinkFunction()();

    console.log(person.drinking) //'eggnog'
    console.log(window.drinking) //undefined

The code up top is also equivalent to the following

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

    var Person = function(){
        this.age = 10;
        this.drinking = '';
        this.drinkFunction = function(){
            if(this.age < 21){
                return function(){
                    this.drinking = 'eggnog';
                }
            }
            else{
                return function(){
                    this.drinking = 'booze';
                }
            }
        }
    }
    var person = new Person();
    //note how even without binds we can set the 'this' object inside the anonymous functions using call
    person.drinkFunction().call(person);

    console.log(person.drinking) //'eggnog'
    console.log(window.drinking) //undefined

I know that even when trying to explain it in the most normal non technical terms possible, there definitely is still a lot of confusion that arises from understanding this entirely. What I highly recommend is to open up your JavaScript console, REPL, inspector, whatever and seriously try creating constructors, functions, functions in functions, things outside of your comfort zone as it relates to this and executing them using binds, calls or applies (which I know we didn’t talk about in the article but its functionality is very similar to call), and see how they affect the object that you bind to. If you want to learn more about binding, consult the documentation for bind, call, and apply. Each are applicable in their own uses which exceed the scope of this guide.

JavaScript can simulate classes, but how do you organize them in files? Native Java can support a single class in a single file as well as any additional subclasses within that file. Different classes are brought together in Java by specifying its dependencies at the beginning of each file with the ‘import’ keyword. Unfortunately at this current state there is no largely supported ‘import’ feature for JavaScript which brings JavaScript developers out of the woodwork to make up functionality for it. RequireJS seems to be a pretty good module loader that emulates the functionality of an import. This means that it’s no longer necessary to rely on properly ordering your dependencies in <script> tags in your html, but rather declare dependencies directly in the JavaScript files themselves. This also means that you can just insert the script tag for just the RequireJS file in the html. This file will be the entry point of the program and simulates the public static void main(String args[]) function for Java. Imports is a pretty awesome feature that JavaScript was missing in creating clean modular object oriented JavaScript.

1
2
3
4
5

    //the equivalent of import Class1; import Class2;
    require(["./class1", "./class2"], function(Class1, Class2){
        var class1 = new Class1();
        var class2 = new Class2();
    });

Okay so now we have a way to ‘import’ js files together and specify dependencies, now we need to see what exactly we expose in these files. JavaScript has ways of hiding information from outside classes but RequireJS also has the option to choose which paramters/functions to expose to the outside world. I’ve been taking a look at the Brackets source code and the approach that they take is to expose public methods and properties through RequireJS’s export object. This works as a method of information hiding but I think the problem with this is that instead of using JavaScripts construct of information hiding by using scoped variables in functions, the decision to create public or private variables/methods is left up to whether the human behind the code wants to expose it. It also changes the traditional view of JavaScript classes meaning someone who is already familiar with JavaScript will have to re-familiarize with how a RequireJS project is set up since methods and variables don’t have to be encapsulated within a class function. Then again there is always a learning curve when an unfamiliar framework or library is used in a project.

RequireJS gives us a lot of options in what to show and what to hide. Traditionally RequireJS allows us to return a single object. This can be a primitive, or a JSON object, an object with properties, or a function. The method that Brackets took is through exposing several properties and objects defined in each JavaScript file through the ‘exports’ object. This method works as you can see from the Brackets source code but it assumes that the methods exposed through this object is from a singleton object. Further internal classes can be made for each singleton object and exposed through RequireJS export object. A good file to look at for an example would be the Menus.js. Checkout the exports at the bottom of the page, it exposes some of the variables and functions defined earlier but also ignores others keeping those private. As a naming convention they prefix these private attributes with an ‘_’. The export also exposes inner classes. These classes are created in a more traditional object oriented JavaScript fashion as in through a function. Check out its api page to get a better idea of the variables, functions, and classes each file exposes.

1

    //placeholder for example require.js code

I know there’s many ways to organize a JavaScript project but I don’t agree with Bracket’s implementation entirely. I think the way that Brackets exports the file as an already created singleton object is something that’s not entirely familiar with people coming in from Java/C# backgrounds. While JavaScript is really good about creating singleton objects, it’d be more familiar for Java programmers to only expose the constructor function for each file. Each JavaScript class would then be created in the traditional Javascript fashion and methods would be public or private based off the standard JavaScript way made possible through function scoping and closure. I think this method of project organization would be more simpler due to people not having to adjust their ideas on what they traditionally know about object oriented JavaScript from Java/C# while also practicing a separation of code.

Here’s a pretty contrived example of what I’m talking about.
source code

So in this program I’m modeling a Car object that takes in a Person object as a driver. These are split into separate JavaScript files. The index.html file has a reference to the require.js file which calls main.js as the program starter. The main.js is where a Car and Person object will be created to model a car with a driver. RequireJS allows us to require Car and Person as a dependency where it is passed into the main program as function parameters. Both car.js and person.js simply return JavaScript class functions where users are free to create objects from. This is the closest setup I can imagine that simulates a classical object oriented program structure.

Here’s the content if you’re too lazy to click on the source link

1
2
3
4
5
6
7
8

    <!DOCTYPE HTML>
    <html>
        <head></head>
        <body>
            <h1>RequireJS Module Example</h1>
            <script data-main="js/main" src="js/require.js"></script>
        </body>
    </html>

Note that main.js imports Car and Person and simulates a person entering a car and going.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

    require(["./car", "./person"], function(Car, Person){
        var civic = new Car("sedan");
        console.log("car type: " + civic.type);

        var person = new Person("Jason");
        console.log("person name: " + person.name);

        civic.go(); //"no response"
        civic.loadDriver(person);
        civic.go(); //"no response"
        civic.start();
        civic.go(); //"vroom"
        civic.exitDriver();
        civic.go(); //"no response"
    });

Note that properties ‘on’ and ‘driver’ are arbitrarily private as an example to showcase that both privileged methods as well as prototyped methods work when imported into main.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

    define(['./person'], function(Person){
        function Car(type){
            var on = false;
            var driver;
            this.type = type;

            this.start = function(){
                on = true;
            }
            this.stop = function(){
                on = false;
            }
            this.isOn = function(){
                return on;
            }
            this.loadDriver = function(person){
                if(person instanceof Person){
                    driver = person;
                }
                else{
                    driver = null;
                }
            }
            this.hasDriver = function(){
                return !!driver;
            }
        }

        Car.prototype.go = function(){
            if(this.isOn() && this.hasDriver()){
                console.log("vroom");
            }
            else{
                console.log("no response");
            }
        }

        return Car;
    });

1
2
3
4
5
6

    define([], function(){
        var Person = function(name){
            this.name = name;
        }
        return Person;
    });

Having this is super handy and something I miss when jumping from Chrome’s javascript debugger that allows you to execute any arbitrary code in the console.

source code
source game

Bejeweled Bot is a bot that automatically plays Bejeweled Blitz as fast as possible. Once executed, the bot will orient itself with the origin of the game board, determine the current gems on the grid, parse the possible matches, then move the appropriate gems in the game to create matches. I forked the repository from kklemm91 and improved on code organization, naming, and execution efficiency.

Execution

Upon initial execution, the bot looks for two identifying pixels indicating the upper left origin of the bejeweled grid. Once found we can then determine the location of all the jewels on the grid. Each jewel is a set pixel width and height apart from each other which makes it easy to sequentially check each pixel color that’s at the center of each jewel. Each jewel color is given an identifying enumerable int and placed into a 2d int array. This array is then parsed index by index to find any potential matches. If a match is found, the program immediately takes action and matches the jewels in the game and then continues looking for more matches.

While gaining a score of 500,000 is considered to be the most impressive feat, the bejeweled bot regularly is able to score greater than a million consistently without assistance.

Issues

Unfortunately, there are moments where the program may stall or softlock. Bejeweled doesn’t use any fancy dynamic filters meaning that checking the same pixel location of the same type of jewel will produce the same result every time. The only exclusion to this rule would be when the jewel is dynamic such as the lightning jewel or the hypercube which is animated to spin producing inconsistent results. These inconsistent results mean that the matcher may miss these matches due to the pixel not being caught and parsed. Multiplier jewels are consistent but not all of them have been documented due to the tedious nature of the work. This means that matches involving multiplier jewels may be missed due to the program not knowing that the multiplier pixel is that of a matching color. This is easily fixable but tedious due to having to play enough bejeweled matches to find the randomized multiplier jewel color. This means that the bot does have tendencies where it stalls due to not being able to detect a match on the board, requiring human intervention so that it can move forward. This form of pixel detection is also dependent on PopCap not changing the art of the game, otherwise the bot will no longer be able to determine the matches.

Instances where the game freezes may also softlock the computer’s mouse due to the game attempting to match a jewel indefinitely. Exercise caution if executing this on a slow machine.

Improvements

While kklemm91’s initial program was a good start I wanted to see if I could make it better. When executed, the initial base program would make all the matches visible in the screen, then wait roughly half a second before making the next set of matches. This was good but I wanted the bot to be inhumanly fast.

In this base program, there was a line that would make the current thread sleep for around a quarter of a second. I noticed that modifying this to be shorter or removing it entirely didn’t make the matches happen any faster. I was wondering what the hold up was in the program so I created a rudimentary profiler where I would log out start and end timings into each important method (I’m pretty sure there’s better ways to profile Java methods). I noticed that most methods ran almost instantaneously but the problem method that was running slowly was the match finder. It wasn’t the act of finding matches that was slow itself, but the fact that it was recapturing a snapshot of the board for each pixel that it checks. This is completely unnecessary and the program was changed to take an initial snapshot, determine all pixel colors from this snapshot, then find matches, based off that board, repeat. This act was like removing the handicap from an olympic sprinter in a children’s fun run. The bot went zooming and was matching at an insane pace.

Another improvement involved the search for the origin point of the board. There was an issue where the program would inconsistently determine the correct origin point. Sometimes it would find it correctly, other times it would offset the origin a few columns to the right, skewing the vision of the bot. Initially the bot determined the location of the origin of the board by detecting whether the pixel was one of the jewel colors. Once found, it then checked to see if several of its neighbors were also has jewel pixel colors. This for some reason produced inconsistent results in determining the origin of the game board and to be honest its absolutely confusing as to why it would do this. I was unable to find the root of the problem so I created a new origin determining method that checks for a sequence of pixels. It’s the same concept as the original implementation but when it’s based off the color of the gems itself, it appears to produce inconsistent results.

An improvement in color matching that would remove our dependency on PopCap not changing its colors would be to use average color detection. Instead of identifying colors based off pixels, we could use an average of the color that falls within the bounds of a block in a grid. This means that the colors don’t have to be pixel perfect so if PopCap decides one day to update the art but keep the same color of the jewels, it’ll most likely continue to keep working. This improvement seems a bit overkill though and is not necessary at the time due to the nature of the project. If I had a bigger budget though I’d gladly like to implement a basic color detector. I’d imagine the performance would take a hit but it would be interesting to see the program dynamically decide similar looking colors.

Well there you have it. Thanks to kklemm91 for showing me the awesome Java Robot class and building out the base of the program.

source code
source game

Text Twist Bot is a bot made to play the anagram solving game Text Twist. Once ran, the bot will auto detect the location of the window, parse the current letters on the board, and automatically input all anagrams that can be found from those six letters. Of course solving anagrams is not new in the field of programming but whatever, revel in its glory!

In its most basic form, the Text Twist Bot is simply an anagram solver wrapped in a robot that detects the game board and inputs the letters. Upon execution, the program reads in an extensive list of english words. These words are placed as a value into a HashMap where their keys are the letters of the word but alphabetically ordered. This means that words like ‘eat’ and ‘tea’ will both have the keys of ‘aet’. You might then ask “But Jason, doesn’t HashMaps only accept unique key values?” then I’ll answer back “Why yes, that’s correct, that’s why I extended the base HashMap class with a MultiValueMap that automatically creates a Collection as a value when new values are added to the same key”. This means that no value with the same key will be overwritten but rather added onto in the collection. Initially I used a MultiValueMap from the apache commons collection library, but after being asked to implement it in an interview (that I bombed) I felt compelled to go back and re-implement it by extending the base HashMap class. Hashing and adding a word to the MultiValueMap takes constant time O(1) for each english word in the list meaning the whole process of adding all the words in the list runs in linear time O(n) where n is the number of words in the list. Now that all the words are processed in the map, I can now lookup anagrams of each groupings of letters at constant speed! Amazing!

Finding the location of the game board and the letters to use was another challenge. The Java class Robot definitely comes in handy when doing these kinds of screen reading actions. To determine the location of the board, I take a full screen picture of the primary monitor where I then go pixel by pixel looking for two specific origin pixels. Once these pixels are found, then I would know the origin of the game board as well as the location of the letters since the letter location is positioned relatively to the origin. I’m still not sure if doing these pixel by pixel searches to find something is the best way to go about it but after seeing how fast it goes about the search, I felt that it was a good enough solution for the problem at hand.

Now that I had the location of the game board I needed to find out the letters to solve an anagram for. Since I didn’t use any OCR technologies for this project, I needed a way to determine which letters were currently on the board. In Text Twist, the letters are represented by spheres. These spheres initially appear on the base row. When pressed, the letter then shifts up to the input row leaving behind an empty yellow sphere. Knowing this, I had the Robot type the alphabet multiple times for each letter in order then press enter. This would sort the letters and bring them back to the base row. I would then slowly type the alphabet in sequence and determine whether a letter was shifted to the input row by using pixel detection. If a letter had shifted, then we know the letter that was just typed is in play. This would continue until all letters were found. The letters would then be alphabetically sorted then passed into a Powerset generator where each set represents a key in the MultiValueMap. Once the values were found, it was game time and the Robot will feverishly type in the letters as fast as the game would take them, only stopping once it finished with inputting all words found.

Issues and Improvements

Of course everything isn’t all sunshine and rainbows. If the robot inputs the text too quickly, the game can’t process it and will ignore it often mucking up an answer and causing the game to stall. Delays were implemented to make the Robot delay each keypress by 15ms. Also, the word list that I found sometimes doesn’t have pluralized versions of the word meaning it usually will find a word but not its plural form. I had a create a cheap word pluraliser that doesn’t cover every instance. Also a word just may be missing from the word list which means I add it in manually after the game lets me know which words I missed. In short the bot isn’t perfect. More work can be put into it to make it have 100% word coverage but I think it does the job pretty well for now.

I’m sure I and many others suffer from this problem. How would we be able to impress when the veterans have already experienced what the newbies have already gone through? How do we know for sure that our thoughts and short amount of experience in our career have any basis or merit? How can we be so confident in an idea that may actually be grounded in ignorance? How can I give this advice to newbies when I’ve only had around three years of professional workplace experience myself? The answer to all these hypothetical questions is who really knows and should we even care so much. When I started my new job, I took the humble path where I primarily just listened and absorbed implementation strategies from meetings and mainly kept my updates straightforward and to the point. After a while my manager recommended that I take communication classes for the annual training. Although I’m not sure exactly the reason why I can only guess that it was because of my misunderstanding of what was expected of me as an employee. I think my manager wanted me to speak up more often even though I still had an incomplete picture of how the company operated as a whole much less how the technical side of the company operated. I convinced myself that I really didn’t have any good insight considering that it was my first year at my first professional position while all my prior experiences were imitations of what proper coding and design should be. Looking back on it, I think that may have been a mistake. I now believe that its important for newbies to open up and provide their insight into whatever the discussion is at hand. If the newbies insight turns out to be dumb and based off ignorance, who cares right? Can the newbie really be at fault for trying? And I believe it lets the veterans know that the new hire is working, as in he’s functional and ticking. Confidence as a newbie is tough when you’re probably the least knowledgeable person in the room but I think speaking out is a better show for the veterans than staying silent.

Of course be cautious of appearing overconfident else you risk looking like a jackass. There’s a small line to be trodden when being confident versus arrogant and honestly I feel that the line is so thin that its truly difficult to know when you’re treading into arrogance territory. I believe as long as you back your assertions with facts then I believe that gives you a right to be steadfast in your ideas. If there are no solid facts in backing your idea then I think it’s better to still suggest the idea than to keep quiet about it but be very open to dismissing it if it turns out to be not great. There was a time during a peer learning session about creating mobile responsive sites where the presenter suggested we use a grid based framework i.e. 960gs, bootstrap. After the lesson, I asserted to only use a grid based framework layout as a prototype and never in production due to how styling the grids usually leads to non semantic class naming requiring maintainers to update both the html content as well as the CSS when needing to update styles. Although not common, site redesigns would require two times as much work and any significant display change of a section of html would always require editing of the html. To be honest, I didn’t feel very good about dismissing the lecturer’s approach. I don’t think anyone really appreciated it. I honestly think I tread the line into asshole territory that day just to get my two cents out there. But I observed something about getting noticed in the workplace, its to always be the loudest. It’s usually the loudest people who share their opinions all the time that get promoted the fastest, that stay in high-ranking positions. As long as the point is valid and relative, it seems beneficial to bring it up even when it might seem like just a complaint or a counterpoint. I never really liked these kinds of people, but the workplace doesn’t seem to benefit those who keep their opinions to themselves. It’s a tough line to tread and I’m eager to try my new ideas of confidence at my next career.

When you actually have a good idea, don’t keep it to yourself, tell everyone. Don’t just share it with your friends, don’t just share it with your manager, don’t just share it with your team, share it with everyone. In order to get more exposure I wanted to propose a bunch of improvements to our figurehead desktop and mobile site. I created a document of several fixes we can apply to the site to get it to operate more smoothly especially in older browsers. All suggestions were prototyped and tested and the results with hard data were shown to improve page draw speeds. I sent the document to my manager where the initiative ultimately died. After that, I learned that it’s not the manager’s responsibility to look into your best interest. I also learned that I should have sent it to everyone, not just my own manager, but the other manager, and to product development, and to the vice president of the tech division, and to anyone who can read. I was a little disappointed that my self proposed project was ultimately dismissed but I wasn’t really happy that I knew I could do something about the terrible rendering of our mobile site and implemented some of the simpler improvements as a tag along to another mobile project. Unfortunately for me and my career, nobody really knew about the improvements except my team. I know now, that if an idea is good, its best to tell everyone.

While humility is best reserved for those who are already respected, its something that new employees in any field should really let go of. There are no opportunities to be found if a newbie is to stay silent, its best to make noise and shake things up.

Regular expressions are hard. There’s a lot to them that vary between different regular expression engines. They’re often considered to be an afterthought when it comes to a programmer’s tools due to the relative infrequency at which they’re used. Whenever they are actually needed, the programmer most likely only reads the minimum amount required that will fulfill the parsing problem at hand. Regular expressions are super cryptic to read and even more so when writing them in Java. While I don’t necessarily enjoy working with regular expressions, I highly respect those who are comfortable enough to take on any sort of parsing problem. This post is an attempt for me to solidify some of the basic and advanced things I’ve been picking up with regular expressions recently.

Greedy will grab more, lazy will grab less

When people keep mentioning use greedy vs lazy quantifiers they’re talking about the ? + * symbols. The default quantifiers will use greedy matching, meaning, they will try to match as many of the quantified character before it continues trying to find matches with the rest of the expression. Appending a ? to any quantifier will cause the regular expression engine to become ‘lazy’ and only match the as little as possible before it finds a complete match. While having a lazy engine may sound like a bad thing sometimes you just don’t want your engine working so hard and going overboard with its matches. Say you’re making a chat client and you want to intercept colon delimited messages so that you can eventually replace those with emoticons.

1
2
3
4
5
6
7
8

    var msg = "hello world :smile: :happy:";
    //greedily match any group of characters between two colons
    var emoteRegExpGreedy = /:.*:/g;
    msg.match(emoteRegExpGreedy); //returns [":smile: :happy:"]

    //lazily match any group of characters between two colons
    var emoteRegExpLazy = /:.*?:/g;
    msg.match(emoteRegExpLazy); //returns [":smile:", ":happy:"]

note how the greedy regexp returns a single value consisting of both the :smile: and :happy: combined into a single match, while the lazy regexp properly returns two values splitting up the smile and happy. The greedy . character matcher kept going until the final : was found at the end of the string, while the lazy .? character matcher was content with ending the first match and the first sight of the : after “smile” and started a new match when it found the first : for “happy”.

Using matched content in replaces with groupings

This is something that so easy but not entirely apparent when initially learning about regular expressions (at least not for me). Surrounding an expression with parenthesis will create a grouping. Groupings are designated in the order that they are written in the expression. Groupings can be designated with ( ). A complicated pattern can be written at the start of the expression and reused as a grouping reference later in the expression. Lets say that we want to catch whenever a user writes duplicate words in the chat box because we’re nice and we want to catch users making silly mistakes.

1
2
3
4

    var msg = "hello hello, how are doing doing? I'm feeling good good";
    //greedily match a full word followed by one or many whitespaces followed by the same exact word.
    var doubleWordRegExp = /(\b\w+)\s+\1/g;
    msg.match(doubleWordRegExp); //returns ["hello hello", "doing doing", "good good"]

Text that’s captured in a grouping can also be spit out to be used in the replacement string. So continuing with our chat emoticon example, lets say that you want to replace the matched : delimited text with an image tag that points to the appropriate file for each emoticon.

1
2
3
4

    var msg = "hello world :smile: :happy:";
    //greedily match word in between colon and put it in a grouping that will be used in replace text
    var emoteRegExpGreedy = /:(\w+):/g;
    msg.replace(/:(\w+):/g, "<img src='$1.png'>"); //returns "hello world <img src='smile.png'> <img src='happy.png'>"

Parenthesis are placed around the \w+ and is accessed with the $1 in the replacement text. Notice how the grouping is accessed in the replacement text with the $ symbol in JavaScript. This symbol differs in different languages from what I understand.

Lookahead and Lookbehind to validate a match

Lookahead and lookbehinds are additions to a standard expression that allow you to check whether the base expression either follows, or is followed by another expression. Lookahead and lookbehinds are not matched, as in they will not be returned by the regexp engine. This is helpful in that sometimes you just want to capture a subset of a complete expression. Lookaheads are performed with (?=suchandsuch) or (?!notsuchandsuch). Lookbehinds are performed with (?<=textbehind) or (?<!nottextbehind). Continuing the chat box example, lets say that we want to implement a feature where users can vote in a poll. Users can vote for a certain option by suffixing ‘vote’ to a number. Our goal is to intercept that number tied to the ‘vote’.

1
2
3
4

    var msg = "I'm going to vote for 5_vote";
    //find matches where a digit is suffixed by '_vote'
    var voteRegExp = /\b(\d+)(?=_vote)/g;
    msg.match(voteRegExp); //returns ['5'];

To be honest the example is contrived. Usually we would want to prepend ‘vote_’ but JavaScriptdoes not support lookbehind so I had to use look ahead in this example.

Okay! That does it. I know there’s way more about regular expressions I gotta internalize but I think this is a good place to end for now.

References and things

RegExper - Regular expressions as railroad diagrams
regexpal - On the fly regular expression matching
Regex Crossword - Crosswords where regular expressions are the hints.