revServer: Spreading the load
or why wise developers use asynchronous workflows
by Andre Garzia
Introduction
Today I decided to talk about how to spread the load of your web application. There has been a lot of discussion lately about memory limitations and execution time limitations regarding RevServer (actually regarding On-Rev). The fact is that all web hosts that use shared servers need some kind of virtualization police to prevent rogue processes from taking all RAM and/or CPU and thus rendering the whole system unresponsive. This kind of policy can be implemented on different OS levels and how each web hosting service does it is beyond the scope of this email.
On-Rev service has a policy of allowing a process to run for about 30 seconds and to take up to 64MB, while this seems small to all the desktop developers in here, all of which are used to swallowing big chunks of memory and CPU (for example people trying to insert 1 GB of data into Text fields), these are actually very sensible values and should accommodate most users. Those 30 secs are like forever in terms of web serving, usually a page takes milliseconds to be served. 64MB of RAM is also a big sum. Here where I work we use the biggest database I've ever seen, it is spread among different machines but the one I am working now has 83 million records totaling 9GB of data and this is our small database, the big one holds more than 7 thousand tables and millions and millions of records. We're in the business of being evil, I mean, we're in the business of sending email marketing and just one of our machines pushes 26 thousand emails per minute. Right now our system is built with PHP (it was built before I came to work here) and we do all this stuff with a memory allocation of 120MB (of which we use about 70MB), so the 64MB allowed by On-Rev appears quite good. Remember this limit applies only to On-Rev service. I am running revServer on my own VPS and I am yet to face any such limits. Other web hosts have different limits and one should not assume that one's own hosting company has no limit.
Many web developers here are just beginning on their path to total web server domination, most are coming from the safe lands of Desktop application design where you are free to do basically anything. You have as much memory as you can swap pages to disk. You can display a progress bar and have a handler execute for minutes without a problem. Those developers sometimes are unprepared to deal with the constraints forced on them by server side programming. They are not unprepared because they are lazy or anything but because they are not used to the "design patterns" of server side programming. You can't think in terms of something when you never ever saw that thing. Without some knowledge base the only way up on the steep hill of server side programming is quite a hard track. So today's topic is on load distribution and asynchronous workflow. Let us first detail the problem.
The Problem
When coding in RevTalk we tend to create commands and functions to do our tasks. It is not uncommon to have some heavy duty handlers to do all our service in one blocking operation. Even when we have multiple small commands and functions each doing its own self contained task, they are usually called by a master command or function that will call each one of them in order. Like this:
command doMyTask
get functionThatUsesALotOfMemory()
commandThatTakesForever it
commandThatUsesEvenMoreMemory it
...
return it
end doMyTask
Even though each line is a single command or function, calling the command doMyTask is a blocking call, all processing is halted until that command finishes. While this is cool on the Desktop it is a problem on the server side. For example, on that command, we have a function called "functionThatUsesALotOfMemory()", if this function reaches the memory limitation policies of your server, there's a chance that your script will be terminated. The command called "commandThatTakesForever" might take more time than the allowed timeout and thus the web request will fail with something like "server is not talking to us error".
What most developers do when faced with code like that is try to optimize the hell out of it. Change the functions to use less memory, change the commands to work faster. Here in the RevTalk community we have lots of tips and tricks for that kind of stuff. We have even measured how long each "repeat loop" form takes and which one is best for each case. But no matter how clever you are, you are still working with blocking code that accumulates memory and that takes time to execute, trying to optimize it until you break the second law of thermodynamics or crashing into the RunRev office with a shotgun shouting "Raise My Memory Limits before I kill someone!" will not be enough. The most elegant solution is to move towards an asynchronous workflow.
The asynchronous workflow
As they say: To understand recursion you must first understand recursion. To understand the asynchronous workflow, you must first understand the asynchronous workflow. When developers are moving from Desktop development to Web development, usually they have a mental picture where the server controls everything and the client interface presented in the browser reflects that control. So code execution happens in order in handlers not unlike that one that I've shown. This approach is OK for most cases but when you have some big handlers that consumes memory and time in a way that will be harmful to the system it is running on then you'll face trouble. When such commands are called, the client will notice some hiccups or latency in their web experience. For example, if the server is executing some code that takes time to finish, the web page that the client is accessing will take more time to load and he will notice. Everyone has had experiences with web applications that when certain stuff is triggered take forever to answer, perhaps because of heavy number crunching or due to long long long code being executed. We can think of a simple analogy that will make sense to all Rev coders here. Usually server side coding is done like blocking URL calls. Until the job is done, it does not return just like loading a URL with the URL keyword, until everything is done, the process is halted. This is the synchronous workflow that we want to get rid of.
Asynchronicity, if that word actually exists (gmail spell checked thinks it doesn't), is like working with callbacks. If I may go back to our libURL analogy, it is like when you use the "load" command with a callback message. You tell it to do something and to tell you when something interesting happens, it immediately returns and you can keep going while things are done in the background.
This kind of behaviour can be implemented on the server side as well. First you need to reverse your mental picture where the server controls the workflow. Think instead that the client interface controls the workflow and tells the server what it should do. Implement the server being able to answer to small job requests, atomic ones. What happens then when the user loads the page or feature is that the client interface calls the server and tells it "hey do the first step" and then it returns and keeps polling the server for status like "did you finish step 1?" much like your kids keep asking "are we there yet?" When the server answers "yes", you ask it "now do step two" and rinse and repeat for all your steps. What this amounts to is that you are spreading your logic code among multiple web requests, not a single process to do all the tasks. Each web request will do something minimal and return, the client interface which is HTML/CSS/JS will maintain a pool or recursive function that calls back the server checking status and commanding that the subsequent steps be executed. This minimizes RAM use per process, makes execution time per process faster and makes your interface more responsive because it is non blocking. Let me illustrate this with a piece of scientifically proved HTML Table:
The Synchronous workflow
| Server | Browser |
| request page
|
|
functionThatUsesALotOfMemory()
|
|
commandThatTakesForever
|
|
commandThatUsesEvenMoreMemory
|
|
return page
| Be happy |
The Asynchronous workflow
| Server | Browser |
| request step 1
|
|
functionThatUsesALotOfMemory()
|
|
Are we there yet? |
| Yes! |
|
request step 2 |
|
commandThatTakesForever
|
|
Are we there yet? |
| Yes! |
|
request step 3 |
|
commandThatUsesEvenMoreMemory
|
|
return page
| Be happy |
See how things are spread out? If you combine that with workers executing in the background such as cronjobs then things becomes even more smooth since you can decouple the heavy commands from web requests. You can communicate with the cronjobs using some shared database records or text files with commands. If we had some kind of background worker set up for the previous example, it would be even cooler, the calls for "are we there yet?" would receive a "no" while the workers executed and when they finished it would finally answer "yes". With an approach like this, you can implement background workers when/if needed without disrupting or changing much of your code since your client side application (the web interface) would already be prepared to use them.
Here where I work we use a similar approach when dealing with the huge marketing campaigns. For example a user will upload a 4 million records email database into the system. This will be placed in a special folder on the server and be flagged as processing. The web interface has a field where is shows the status of the job, it will show "processing" and will pool every couple seconds to see if it was updated. Some cronjob will notice the file lying there alone and helpless in that folder and will pick it up, and parse it for all kind of stuff and when happy insert it into the database. This takes a while, after it is done it will change the status field for that job to "complete". The web interface will keep pooling and showing "processing" until it is "complete" when it will change the message and enable some controls to fiddle with that database. The interface feels very responsive and the user can do other tasks while that job is processing with no problem since the heavy task is decoupled from the web request. memory wise we only chew what we can actually swallow, our cronjobs are smart and even as they work in parallel, they signal each other and we don't allow memory use to go up like mad. In the end we have a smooth system that does some heavy stuff while it appears quite simple and fast to the end user.
If there's interest in this community I can craft some real RevServer scripts showing this approach. This is the key to being able to serve lots of request while doing intensive work and being a good shared server citizen.
Hope you guys enjoyed the article.
Andre
|
About the Author
Andre Alves Garzia is 30 and lives in Brazil. He is usually coding something network related and created RevOnRockets. When not working, he enjoys SF Books, sailing and making pizza.
|
| 
|
