Moving an ecommerce site to Amazon Web Services

This is an interesting one.. There is an e-commerce startup, run by a dear friend.  One fine day we were chatting around and he mentioned challenges with his business. Challenges included performance, scaling and cost issues about his hosting services provider.

My instant reaction was, why dont you move to aws.. and it clicked just like that.. We decided that we'd start with the free tier, with his mysql server on amazon RDS and rest of app server functionality on the micro instance. All of which is within purview of the free tier.  Using Cloudfront we'd localize the static content, to add to the performance of content delivery. Not now, but eventually, we'd start using ELB - the load balancer for distributing load across the instances we would have.

With something like aws, planning all that is really really that straightforward.  You just have to be a bit more technically oriented to think in terms of these things and you are set for good.

We are already thinking about configuring autoscaling for the services, so that running on the micro instance doesnt become a bottleneck for the customers. With that we'd hope to achieve dynamic automatic scaling up and down the infrastructure.

We have already taken the first steps, and I am helping his venture migrate to AWS, one of my dream jobs, to be able to consult around migration / integration of Traditional systems to cloud based systems.

writing for cloudtimes.org these days

Read some of my content at http://cloudtimes.org/author/raghav/

AWS Application Demo

• Amazon started with S3 (Simple Storage Service) - purely storage service.
• For Computing purposes, Amazon started EC2 
• EC2 allows hosting your application on virtual servers operated by Amazon, widely known as AWS.
• For calculating the availability as a resource, 1 compute unit at Amazon EC2 is roughly equivalent to a 1.2 GHz Xeon server.

• All computing resources are virtualized, none is physical. No physical details are ever published. However, the user has to manage his own file system. Based on need/request, disk space is mounted and is made available to the server. Further, it has to be managed by the application.
• There can be three types of instances, 
• Small
• Medium
• Large instances

Behavior on instances

• like a normal web server
• has a public web address

• Java Software available to convert command line instructions into SOAP wrapped API calls to AWS
• Need to download API tools from AWS
• Key pairs are tied to regions
• The benefit of this is that the servers are replicated across data centers located physically separately

Autoscaling service

• Allows scaling of servers based on need
• Consumes about 60-90 seconds to scale up based on the need (e.g. load on the server). If configured the appropriate way, the system will replicate the application code and launch new server instances within seconds to handle the extra load.
• Handles scaling up as well as down both at the same time. Automaticaly, the extra/unused instances will be shut down and released from the application deployment when there is no load to require their service.
• very useful in cases of spikes, specially high-peaks. New feature launches on sites, or sudden outbreak of news etc, cause spikes in server loads. At such times, AWS works perfectly to scale up the required computing power. The application owners are saved from buying extra servers for that 1 day load... :)
• Its possibel to say how many servers to add
• Such a scaling up/down can be configured. For example, it can be specifiied that request for new instances when current system's load exceeds 85% or scale down, when the usage goes below 20% used

OSs supported

• Linux
• Windows
• openSolaris

Buzzwords

• AMI = Amazon Machine Images
• EBS = Elastic Block Device
• Elastic IP = static ips assigned to instances
• Amazon EC2 = Amazon Elastic Computing Cloud
  

The workshop day @GIDS 2010 - I - cloud computing using Google App engine

Well, A developer day cant get better than that... And, whatever issues I mentioned regarding scheduling of the summit were gone up in air.

I had the choice to go attend the Cloud Computing/hosting workshop with Mathhew McCullogh. It was based on java, which I dont care about anyway. Not that I have any kind of disrespect for the language, but I strongly think that we ought to think beyond languages.

He perfectly showcased how easy it is to build and host you own web application on the google app engine cloud platform. Its hard to imagine how easy it has become with eclipse plugin for google app engine and web toolkit, to develop an application for the cloud and deploy it. With those tools installed and configured (its surprisingly easy task to do it anyway), its a button click to deploy your application to the cloud.

I am not a big fan of java, or of huge complex configuration files for that matter, but the way this app engine thing is coming pre-configured, it reminds me of ruby on rails. At least to start with. Its surprisingly easy to start developing with GWT (Google Web Toolkit) and Google App Engine.

I tried to replicate what he was doing/showing, and, remember I am no big shots in java, that last professional java work I did was about 8 years back, It was surprisingly easy. Just a few clicks and it was done. I have to say, it was infectious, as if I felt like delving more into java and go back to experimenting with it...

Is that a workable idea ?

Actually, we are waiting for a new server to be installed and any new physical server installation takes some time, we all know that. The project would request it, there would be financials and then the order would be approved internally.
It would be placed with the vendor by purchasing, the vendor will ship it, and then a few days/weeks after, technicians will come over to install the stuff.

Then the local admins will get to work and configure it to be used by the respective teams. Another few days/weeks...

Just wondering, any given enterprise has hundreds ir not thousands computers, all classes, laptops, desktop development PCs, server class PCs and others...

How about taking off 10% of all these PCs' computing power to create a cloud computing environment (of course within the company environment only). An application can be installed which will make sure that a given percentage of computing power will go to only a particular resource.

A lot like how BOINC works. You install a client/manager app on ur PC and then u receive a piece of computation for your part of computing power.

That kind of computing environment can easily replace an average server's computing power.
The storage can easily be hooked up to a SAN.

I'd encourage a discussion on this... see what others think around the idea...

in reference to: Distributed computing - Wikipedia, the free encyclopedia (view on Google Sidewiki)

Discovered BOINC 5 years ago, joined today :)

It was George, a colleague at Axa Tech in Switzerland who first introduced me to this idea. I was thrilled alright, but didnt join right away.

Today I did.

Its a nice feeling, and a feeling of giving back something to science.

http://en.wikipedia.org/wiki/Berkeley_Open_Infrastructure_for_Network_Computing

http://boinc.berkeley.edu/

I would not go into details how it works and all that. However, I would have to say, for whatever time your laptop/workstation remains idle, you can donate the processing power to a better cause. Absolutely fantastic idea, genuinely noble.

And then, look a the stats, people like me are contributing to the processing power generated to the extent of 4 pFLOPS, as against the fastest supercomputer system at mere 1.79 pFLOPS.

As I said, feels great. :)

I hope to inspire others to join in...

in reference to: BOINC (view on Google Sidewiki)

Earning money using cloud Computing ??

I dont know much more about cloud computing, but it sure sounds like an interesting idea. The way it allows computing resources to be shared and connected through internet and all that. The benefit to the average developer is huge, getting computing power so cheap..

However, I think the reverse should also be true and could turn out to be pretty interesting. The way we (developers) use resources from the cloud for our computing purposes, we could also share our spare resources to the same cloud so that someone else might use them and allow us to earn some money in exchange. There would be some preconditions and requirements and what not, but essentially that should be a possibility...

I dont know for sure whether this concept already exists, but i would sure like to see that happening...