F1 Torrents of the Legal Kind
We talked briefly on last weeks show about some minor issues we’ve had recently in regards to our bandwidth costs. If you managed to get to the end of that episode we mentioned a spike in traffic led our hosting company, Media Temple, to warn us we could be looking at a bill for $350+ at the end of the month, and then the traffic spiked yet further.
The Price of Fish
As you can imagine, there’s no way we could afford those kind of costs once, let alone on an ongoing basis. The hosting company includes in their grid-service, one terabyte of bandwidth per month and until now that’s suited us just fine. Anything over that though gets billed at more than two dollars per gigabyte and in July we shifted almost double our allocation.
Media Temple do a sterling job of providing us with solid web hosting, the kind that lets us handle 1,600 comments per thread without blinking, but they never claimed to offer infinite bandwidth.
We’ve obviously been testing out a bunch of alternate solutions and amongst them is Amazon S3, which in truth has been a little unreliable of late, but does offer unlimited bandwidth (although it is expensive). More importantly though, S3 acts as a tracker for torrent files as well as a permanent seeder and this is where things get interesting.
Getting Personal
Torrents, BitTorrent and peer-to-peer file sharing tends to come in for a lot of criticism in the press, mostly when record companies sue pensioners and toddlers for alleged music piracy. However, the concept of sharing data between multiple peers has many legitimate uses, BBC’s iPlayer is based on the same principles, and it’s the legal distribution of data that we’re interested in.
Making our audio and video available for download via peer-to-peer networks, in theory reduces our bandwidth requirements to almost nothing, as clients share pieces of data amongst themselves (at least for anyone obtaining files using this method). Additionally, if no-one is sharing a particular file, as will likely be the case at first, then the S3 peer will always be available.
It’s easy to see why we’re fans of P2P, and we’ll be promoting it more in the future. It’ll take a while to copy everything we’ve ever created onto to the bookseller’s servers, but thus far torrent files are available for:
- All video files
- Every episode of F1 Digest
- All interviews
- Most of the mini-series
- Recent episodes of the weekly audio show
Additionally we’re adding links on individual blog posts alongside the familiar transcript shortcuts and standard file downloads.
We’re not expecting this to make a huge difference at first, and neither are we removing any of the existing download options. In fact, unless you’re feeling brave or are specifically interested in testing these new download options, things will stay exactly the same.
As ever we’d appreciate any and all feedback, let us know if works for you or if there’s anything we can improve.
August 10th, 2008 at 1:41 amBits and Pieces | Sidepodcast : Your Weekly F1 Podcast said:
[...] up on an earlier post relating to torrent support at Sidepodcast, I figured it might make sense, having covered the reasons for using peer-to-peer technology, to [...]
August 10th, 2008 at 7:11 pmKris said:
I have the misfortune to be unable to connect out to the internet at large. What I do have, however, is access to a proxy server that is happy to go fetch any page I ask for, and deliver it to me (as long as i don’t try using https on anything other than port 443 - can’t have me tunnelling of course
)
iplayer (both the flash based streaming version and the kontiki based download version - same for 4od) both simulate standard http on port 80/8080 (and sometimes others) to facilitate downloads, which means that they “just work”.
I’m able to use some convoluted tunnelling to get torrent traffic etc through in an emergency, but that isn’t really practical for regular use.
Although using S3 as an always on seed & tracker for a swarm of torrents is insanely cool (I’m a geek… so what!) its just not going to work for some of us.
My proposed solution is nice and simple. Set up another web hosting account with someone who offers another good chunk of bandwidth, dreamhost for example offer 5TB/mo (with 500gig disk space) for between $6 and $11 a month depending on how long you pay in advance. Set this account as media.sidepodcast.com and use that for the audio and video.
Although this might not be as cool (and perhaps not as quick) as S3, its likely cheaper and simpler.
If at some stage even that becomes insufficient, add another account (either at the same place or elsewhere) and use a DNS round robin to distribute the load
August 10th, 2008 at 7:22 pmme said:
that’s okay kris, we’re not going to stop offering direct downloads, nor are we going to discourage browser streaming.
in terms of reducing our monthly costs S3 on it’s own isn’t going to help in the slightest, we’re just playing around with our new found ability to torrent everything
yeah, we looked into that, but man those kids are slow when it comes to big file downloads.
in terms of cost savings, we’re currently splitting the difference between blip.tv and libsyn. we’re experimenting with other services too.
ummm, you’ve out geeked me there. how’s that work?
August 10th, 2008 at 8:05 pmKris said:
Really? In my experience its been file writes, and CPU intensive tasks that the big shared hosts struggle with..
In fact, I just did a highly unscientific test with my own account, the third ep of forgotten f1 teams (3.9M) downloaded in 6 seconds, whereas it took 10 seconds to come from this server.
If you’re interested, I can send you a link to the file for you to conduct your own equally unscientific tests
If its going to be too slow, then its something of a non-issue.
Round Robin DNS works by having the DNS server cough up more than one (optionally weighted) address for a given hostname, clients pick one to connect to, - hopefully not all the same one
August 10th, 2008 at 8:52 pmme said:
if you could, that would be appreciated. last time we tried it with a large video file and it seemed to take forever to come down.
am with ya. we may need to look into this.
August 10th, 2008 at 10:44 pmKris said:
Email sent to christine: I don’t know your email address.
August 10th, 2008 at 11:18 pmAlianora La Canta said:
Email sent to christine: I don’t know your email address. {Kris - previous post}
If you don’t know Christine’s email address, how will it reach her? Or am I just too tired to see the obvious?
August 10th, 2008 at 11:23 pmme said:
no, no. christine got the email destined for me. she’s now forwarded it on
August 11th, 2008 at 3:50 amJordan Allen said:
Use “avian carrier”, unless “me” has installed the “Talon” carrier removal hardware.
August 13th, 2008 at 12:25 amRich said:
I am glad you will still provide service to those of us that have service providers that have lock down any P2P traffic - the same reason I cannot Skype you guys.
August 14th, 2008 at 10:42 pmDave Monks said:
Would RapidShare (RS) offer you a better service than S3, or is it all very much of a muchness?
The extra difficulty from the client end with RS may be off putting - and having a true ‘3rd party’ feel to accessing the sidepodcast media may draw away from the professional nature…
August 14th, 2008 at 11:27 pmme said:
at the moment, we have both blip.tv and libsyn running tandem. we’ll watch both of them and see how they fair over the next month or three.
blip is preferable as they’re cheaper and libsyn have proven themselves to be all shades of useless in the past. but libsyn downloads noticeably faster. we’re considering every possible option mind you
we’re also looking into the round-robin dns thing too (well i say “we”)