Technical Mumbo Jumbo

I am going to take a brief hiatus on discussing game mechanics to discuss some of the technical solutions that helped us get to where we are now.

This game runs on a couple of key technologies, some of which have emerged just in the last few years.  The first key technology is Java 2 Platform for Enterprise (J2EE) including the full servlet stack using Tomcat as our servlet container.  The second is HTML5 including the latest JavaScript.  These two technologies make up the backbone of our entire project, J2EE for the back-end server code and HTML5 for the client side code.

Originally, (around 2014) the back-end code relied on a PostgresQL database directly and this worked up until we needed to update millions of resource entries per second, at which point even the batched groupings of SQL statements, like,

UPDATE schema."TABLE_NAME" SET "RESOURCE"="RESOURCE" + "RESOURCE_CHANGE";

were still not fast enough to update all of the resources for every planet, since it was responsible for updating millions of entries.  This project nearly died here were it not for the clever thinking of one of our previous technical people, Cris, who later moved on from the project.  He suggest the idea of using an in-memory caching system, which ended up solving the speed problem completely.

We began by looking at existing caching solutions, such as memCacheD, Java Caching System, EhCache, and Google Guava, but we discarded them for a variety of reasons including excessive overhead, poor output formats, uncertain security, and poor performance in our cloud environment.  Ultimately, I ended up designing and building a customized caching system for this game specifically seeking to accomplish our odd set of goals of simplistic structure, ease of Json output without excessive transmutation, security, and speed.

The system revolves around storing a minimalistic view of the database in a Java object in the RAM of the server.  Surprisingly, the entire database in Java only amounts to about 1.6 Gb of RAM usage for 40,000 stars.

The system has the convenient property of allow us to output JSON without creating any new Java objects for most of the Application Program Interface (API).  The API covers all of the communication protocol between the game client and the server and most of the communication is done with HTTP GET requests.

The game client itself is an HTML5 abstraction that was created using the game development software Construct 2.  This program allowed us to quickly get a demonstration test client up and running displaying 100,000 stars immediately, so we selected this suite over other options like GameMaker, D3, or doing it in raw JavaScript.

So, looking at it altogether, the game stack looks something like this:

The PostgresQL database provides us with failure resistance, the Tomcat servlet container allows us to run all of the multi-threaded servlets and game engines that crunch numbers quickly, all of which are stored in the in-memory cache and fed to the game client via the servlet API.  The actual architecture is a little more complicated than this, but I don't want to bore you all.

I'll address other technical aspects in future posts.

á na márië,
gumshoe, out.