How I Cut My Hosting Costs by Moving from AWS to a Raspberry Pi

I've been running my web scraping service (obzrv.it) for quite some time already. I've had a lot of fun working on it, mastered my Go coding skills and learned a lot about web scraping, monitoring and automation.

Sadly, I found it super difficult to attract any paying customers. I was spending $70 per month on AWS EC2 instance. The thing with web scraping is that - to make it really work - you need to open web pages in a real browser, typically Chrome because it offers nice controlling & debugging features. And that's not cheap 😢 Chrome needs resources, mostly RAM - 4 GB is a minimum.

This meant I needed a machine with 8 GB of RAM and ideally 4 cores, so more complex pages render quickly. Speed also matters, because it's opening multiple pages, multiple times a day. That costed some money, and I had to shut the service down after around a year, to avoid further losses.

Meanwhile, while working on another project, I was playing with a technology called port-forwarding. In short, it allows you to expose your local computer to the Internet, in a way that your home PC accepts connections coming from the Internet, just like a real server machine from a hosting provider. You run a small SSH command on your computer (Secure SHell, a way to securely connect to a remote machine, and a standard tool for every developer), connect to a remote server and tell it to forward all traffic from a specific port to another port on your local machine. Technically, that remote computer is your "website" - or rather a "gateway" - but it doesn't really handle users requests, instead it forwards all incoming traffic to your local machine, as if the user connected to your home PC. The big difference in my case was: because just forwarding the traffic doesn't require much resources, neither CPU nor RAM, I could run my web scraping service again, very cheaply 😊

So I did. I ordered the cheapest server available on AWS, a t4g.nano for $0.0042 per hour - that's a whooping $3.02 per month. Now that's the price I can pay for hosting 😎 At first I thought I'd run the service on my wife's old Sony VAIO laptop. It's a 10 years old machine, it has 4 GB of RAM and an i5 processor. Not the fastest, but enough for my needs. It also has a battery, which will keep it running in case of power outage. Finally I decided to buy a little Raspberry Pi 5 for around $100, which is a small computer, size of a credit card, with 8 GB of RAM and 4 cores - also enough to run my service. The entire device costs slightly more than a monthly AWS bill I payed so far.

I saw few advantages of Raspberry Pi over the laptop:

• it's much smaller and I can put it anywhere - and I don't have a spare server room
• it's very quiet as it doesn't need a cooling fan
• it's more energy efficient. When idle, it consumes around 1-2 Watts of energy, a small fraction of a LED bulb. Save money, save the planet!

All I had to do next, was to run SSH command on the Raspberry, which connects with the AWS "server" and tells it to forward all traffic from port 80 to the Raspberry Pi. And of course start my web scraping service on the Raspberry. And that's it! I have a website running on a Raspberry Pi, for $3.02 per month. There are few more technical details, like connecting the domain and SSL certificate, and also making SSH connection persistent, so it reconnects automatically when either the power or the network goes off for a while, but that's a topic for another blog post.

So currently, when you open the web page, it really connects to the Raspberry, which generates the page. And when you run a web scraping job, I can almost feel the temperature raising in my room, as that little CPU gets hot very quicly. Thank you for that 😊

I must admit that web page rendering is a bit slower than on the AWS server, but it's still acceptable. Raspberry Pi is equipped with ARM processor and while you can run 99% of the software on it, it's still a mobile processor, hence optimized for power effciency, not for performance. But what makes a huge differece is the cost - because now I'm paying a constant amount of $3.02 per month, I don't have to worry about the costs anymore, and I can offer more services for free to attract more users and further promote my service. Overall, it might not be a good idea to run a high traffic website on your local machine, but I think it's perfect approach for small projects, startups, personal websites and proof of concepts. It's also great for test or demo environments - it only takes couple of minutes to set it up - customize your url, run the SSH command on your laptop, and you can send a link to your customers or testers, so they can see your work in action, while you can observe logs and debug on your dev machine. No need to setup and maintain (and pay for!) a separate server environment for that.

Finally, I also realized that because port forwarding consumes very little resources, I could handle hundreds of machines like my Raspberry, each running it's own service, and all of them would be accessible from the Internet, for a fraction of a cost of a single AWS instance. And that's how InstaHost was born. I hope you'll find it useful, and I'm looking forward to hearing your feedback!

Check out our blog for more examples of how you can benefit using InstaHost