The Birth of the "FReepCast"? (BitTorrent Broadcasting)

Attention FReepers: If you aren't aware of BitTorrent, it is a software program that allows MASSIVE files to be distributed over the internet at tens to hundreds of times faster than what would be normally seen. Currently it is being used to steal software and media, but the real promise is MEDIA DISTRIBUTION.

A graduate student at Stanford used it to distribute the liberal flick "Outfoxed" to thousands of people over the internet with ease.

The Graduate student distributed the equivelant of 750 GB of bandwidth for 5 GB. That's an astounding 15,000% increase!!!

I PROPOSE THE CREATION OF A "FREEPCAST". We will be able to create our own media network for an extremely low cost. And the great thing is that we will be the first ones to it.

I've already bought the rights to FReepCast.com. I will contribute the rights of the name for free to FR & co. if we can get this done.

I need volunteers who have the geek skills to get it off the ground and I need the media-saavy folks who get create materials!

What is BitTorrent? BitTorrent is a protocol designed for transferring files. It is peer-to-peer in nature, as users connect to each other directly to send and receive portions of the file. However, there is a central server (called a tracker) which coordinates the action of all such peers. The tracker only manages connections, it does not have any knowledge of the contents of the files being distributed, and therefore a large number of users can be supported with relatively limited tracker bandwidth. The key philosophy of BitTorrent is that users should upload (transmit outbound) at the same time they are downloading (receiving inbound.) In this manner, network bandwidth is utilized as efficiently as possible. BitTorrent is designed to work better as the number of people interested in a certain file increases, in contrast to other file transfer protocols.

One analogy to describe this process might be to visualize a group of people sitting at a table. Each person at the table can both talk and listen to any other person at the table. These people are each trying to get a complete copy of a book. Person A announces that he has pages 1-10, 23, 42-50, and 75. Persons C, D, and E are each missing some of those pages that A has, and so they coordinate such that A gives them each copies of the pages he has that they are missing. Person B then announces that she has pages 11-22, 31-37, and 63-70. Persons A, D, and E tell B they would like some of her pages, so she gives them copies of the pages that she has. The process continues around the table until everyone has announced what they have (and hence what they are missing.) The people at the table coordinate to swap parts of this book until everyone has everything. There is also another person at the table, who we'll call 'S'. This person has a complete copy of the book, and so doesn't need anything sent to him. He responds with pages that no one else in the group has. At first, when everyone has just arrived, they all must talk to him to get their first set of pages. However, the people are smart enough to not all get the same pages from him. After a short while they all have most of the book amongst themselves, even if no one person has the whole thing. In this manner, this one person can share a book that he has with many other people, without having to give a full copy to everyone that's interested. He can instead give out different parts to different people, and they will be able to share it amongst themselves. This person who we've referred to as 'S' is called a seed in the terminology of BitTorrent. There's more about the various terms in a later section.

How does BitTorrent compare to other forms of file transfer? The most common method by which files are transferred on the Internet is the client-server model. A central server sends the entire file to each client that requests it -- this is how both http and ftp work. The clients only speak to the server, and never to each other. The main advantages of this method are that it's simple to set up, and the files are usually always available since the servers tend to be dedicated to the task of serving, and are always on and connected to the Internet. However, this model has a significant problem with files that are large or very popular, or both. Namely, it takes a great deal of bandwidth and server resources to distribute such a file, since the server must transmit the entire file to each client. Perhaps you may have tried to download a demo of a new game just released, or CD images of a new Linux distribution, and found that all the servers report "too many users," or there is a long queue that you have to wait through. The concept of mirrors partially addresses this shortcoming by distributing the load across multiple servers. But it requires a lot of coordination and effort to set up an efficient network of mirrors, and it's usually only feasible for the busiest of sites.

Another method of transferring files has become popular recently: the peer-to-peer network, systems such as Kazaa, eDonkey, Gnutella, Direct Connect, etc. In most of these networks, ordinary Internet users trade files by directly connecting one-to-one. The advantage here is that files can be shared without having access to a proper server, and because of this there is little accountability for the contents of the files. Hence, these networks tend to be very popular for illicit files such as music, movies, pirated software, etc. Typically, a downloader receives a file from a single source, however the newest version of some clients allow downloading a single file from multiple sources for higher speeds. The problem discussed above of popular downloads is somewhat mitigated, because there's a greater chance that a popular file will be offered by a number of peers. The breadth of files available tends to be fairly good, though download speeds for obscure files tend to be low. Another common problem sometimes associated with these systems is the significant protocol overhead for passing search queries amongst the peers, and the number of peers that one can reach is often limited as a result. Partially downloaded files are usually not available to other peers, although some newer clients may offer this functionality. Availability is generally dependent on the goodwill of the users, to the extent that some of these networks have tried to enforce rules or restrictions regarding send/receive ratios.

Use of the Usenet binary newsgroups is yet another method of file distribution, one that is substantially different from the other methods. Files transferred over Usenet are often subject to miniscule windows of opportunity. Typical retention time of binary news servers are often as low as 24 hours, and having a posted file available for a week is considered a long time. However, the Usenet model is relatively efficient, in that the messages are passed around a large web of peers from one news server to another, and finally fanned out to the end user from there. Often the end user connects to a server provided by his or her ISP, resulting in further bandwidth savings. Usenet is also one of the more anonymous forms of file sharing, and it too is often used for illicit files of almost any nature. Due to the nature of NNTP, a file's popularity has little to do with its availability and hence downloads from Usenet tend to be quite fast regardless of content. The downsides of this method include a baroque set of rules and procedures, and requires a certain amount of effort and understanding from the user. Patience is often required to get a complete file due to the nature of splitting big files into a huge number of smaller posts. Finally, access to Usenet often must be purchased due to the extremely high volume of messages in the binary groups.

BitTorrent is closest to Usenet, in my opinion. It is best suited to newer files, of which a number of people have interest in. Obscure or older files tend to not be available. Perhaps as the software matures a more suitable means of keeping torrents seeded will emerge, but currently the client is quite resource-intensive, making it cumbersome to share a number of files. BitTorrent also deals well with files that are in high demand, especially compared to the other methods.

16 December, 2003 | Broadcatching with BitTorrent Or, RSS meets BitTorrent meets TiVo. The other day, Steve Gillmor wrote about BitTorrent and RSS and how they could be combined to create a “disruptive revolution.” He’s half right. RSS and BT are indeed two great tastes that taste great together, but Gillmor’s vision is upside down: we shouldn’t use BitTorrent to carry RSS, we should use RSS to carry BitTorrent. Let me explain. But first, some background. RSS (RDF Site Summary) is a simple format for syndicating content on the web. These days, the most common application of RSS is subscribing to weblogs: you tell your computer to check an RSS file for changes every so often, and then it notifies you when there’s something new to read. If you’re like me and you read one metric shitload of news every day, this is a life-saver. BitTorrent, the brainchild of Bram Cohen, is the current cool-kids’ P2P program. It works sort of like Kazaa, but at a lower level. It doesn’t handle searching for new files, it doesn’t have a media player, it just concentrates on downloading big files efficiently. Okay. Two solutions in search of a problem. Here’s a problem: I have a weakness. I am addicted to the show Alias. I watched the first couple episodes of season two as it aired, and I was hooked. In my honest moments, I’ll admit that the show’s appeal is mostly due to the callipygian Jennifer Garner. It’s a weakness; we deal. But it gets worse. I go out on Sunday nights, when Alias airs, and I don’t want to give that up. That’s why God created the VCR, I know, but to compound the problem, I don’t have TV. I don’t want to have TV, because I love the feeling of superiority that I get by not having it. This system is at tension, it has no rest, its forces are unbalanced, it wants to be resolved. A partial answer. The internet, it turns out, is great at resolving different kinds of tensions, and this is one of them. After a few weeks of missed episodes, I realized that with a little patience, a P2P program like Kazaa was able to fetch back-episodes with aplomb. Each file is around 450 megs, fairly high-quality video, with commercials cut out. I start a few episodes downloading, and by the next evening, they’re ready to watch, whenever I have the time. After a few weeks of enjoying this, a new tension emerged: I had caught up with all of the old episodes, and I had to wait a week for each new one. The problem is that the Kazaa protocol isn’t especially well-tuned for getting brand new files: first someone has to record the show as it airs, cut out the commercials, and compress it to a reasonable size, then seed it on the network. Then, it has to slowly propagate to its peers, each transfer taking hours. It might take three days before it’s available on enough peers that I’m able to even find it, let alone download it. BitTorrent to the rescue. The solution is BitTorrent. BitTorrent operates on similar principles to Kazaa, but it’s tuned differently: it excels at downloading files that are new or currently in high demand. It breaks large files into many small chunks, and coordinates their assemblage, so that users can tap into a swarm and distribute the load evenly. At the same time that you’re downloading a chunk, another user is downloading an earlier chunk from you — no one server is overwhelmed, and the more popular a file, the higher its availability is. It’s perfect for large files that are most interesting when they’re fresh — in other words, it’s perfect for TV shows. In many cases, I have been able to use BitTorrent to completely download a new TV show mere hours after the show airs. Like a TiVo user, I’m no longer bound to a specific time to watch my shows. I’m free to go out on Sunday night and still watch my show while it’s brand new. TV is now asynchronous. Life is good. But it could be better. RSS has revolutionized the way I consume some types of content, like weblogs and news. Instead of compulsively reloading a huge list of bookmarks looking for updates, my RSS reader scans them all every hour, and summarizes what’s new — allowing me to keep current on more sources with far less effort and waste. Imagine if I had the same efficiency retrieving video files. With the addition of RSS, BitTorrent could really be taken to the next level, and I’d be able to forget about the plumbing of TV altogether. I want RSS feeds of BitTorrent files. A script would periodically check the feed for new items, and use them to start the download. Then, I could find a trusted publisher of an Alias RSS feed, and “subscribe” to all new episodes of the show, which would then start downloading automatically — like the “season pass” feature of the TiVo. You know, for kids. Illegitimate uses of this system would obviously abound. But the potential legitimate uses are huge as well. For one, traditional content providers (like the TV networks) could take advantage of the demand for their programming by scooping the copyright infringers. If ABC released Alias on BitTorrent with advertising built in, the file could be delivered to their audience very fast, and would cost them next to nothing in distribution costs. The economics of producing video programming would be upended — each viewer of the program would, in effect, foot the bill for a tiny slice of the distribution overhead, causing a massive component of traditional media company infrastructure to become obsolete. The point that Gillmor misses is that RSS files and JPEG graphics are small, and BitTorrent’s overhead makes it ill-suited for small files. The BT/RSS combo is potent, but its potential lies in using each technology’s strengths — RSS for syndication: low-overhead references to fresh content — and BitTorrent for the heavy lifting: swarming transport for high-demand, big files. The beauty is that this synergy can so easily be realized. You can start publishing RSS feeds of BT files today: just point to a .torrent file in the element. A user who consumes the feed in their favorite RSS reader will be able to start the BitTorrent download with one click. Then, as the creators of RSS software catch on, they can automatically start the BT program when they encounter a new .torrent link. The result: the TV distribution networks are completely end-run by an ad-hoc, decentralized, loosely-coupled network. And in the process, significant opportunities are afforded to independent content producers of audio and video to reach a mass audience with insignificant distribution costs.

16 December, 2003 | Broadcatching with BitTorrent

Or, RSS meets BitTorrent meets TiVo.

The other day, Steve Gillmor wrote about BitTorrent and RSS and how they could be combined to create a “disruptive revolution.” He’s half right. RSS and BT are indeed two great tastes that taste great together, but Gillmor’s vision is upside down: we shouldn’t use BitTorrent to carry RSS, we should use RSS to carry BitTorrent. Let me explain.

But first, some background.

RSS (RDF Site Summary) is a simple format for syndicating content on the web. These days, the most common application of RSS is subscribing to weblogs: you tell your computer to check an RSS file for changes every so often, and then it notifies you when there’s something new to read. If you’re like me and you read one metric shitload of news every day, this is a life-saver.

BitTorrent, the brainchild of Bram Cohen, is the current cool-kids’ P2P program. It works sort of like Kazaa, but at a lower level. It doesn’t handle searching for new files, it doesn’t have a media player, it just concentrates on downloading big files efficiently.

Okay. Two solutions in search of a problem. Here’s a problem:

I have a weakness.

I am addicted to the show Alias. I watched the first couple episodes of season two as it aired, and I was hooked. In my honest moments, I’ll admit that the show’s appeal is mostly due to the callipygian Jennifer Garner. It’s a weakness; we deal.

But it gets worse. I go out on Sunday nights, when Alias airs, and I don’t want to give that up. That’s why God created the VCR, I know, but to compound the problem, I don’t have TV. I don’t want to have TV, because I love the feeling of superiority that I get by not having it.

This system is at tension, it has no rest, its forces are unbalanced, it wants to be resolved.

A partial answer.

The internet, it turns out, is great at resolving different kinds of tensions, and this is one of them. After a few weeks of missed episodes, I realized that with a little patience, a P2P program like Kazaa was able to fetch back-episodes with aplomb. Each file is around 450 megs, fairly high-quality video, with commercials cut out. I start a few episodes downloading, and by the next evening, they’re ready to watch, whenever I have the time.

After a few weeks of enjoying this, a new tension emerged: I had caught up with all of the old episodes, and I had to wait a week for each new one. The problem is that the Kazaa protocol isn’t especially well-tuned for getting brand new files: first someone has to record the show as it airs, cut out the commercials, and compress it to a reasonable size, then seed it on the network. Then, it has to slowly propagate to its peers, each transfer taking hours. It might take three days before it’s available on enough peers that I’m able to even find it, let alone download it.

BitTorrent to the rescue.

The solution is BitTorrent. BitTorrent operates on similar principles to Kazaa, but it’s tuned differently: it excels at downloading files that are new or currently in high demand. It breaks large files into many small chunks, and coordinates their assemblage, so that users can tap into a swarm and distribute the load evenly. At the same time that you’re downloading a chunk, another user is downloading an earlier chunk from you — no one server is overwhelmed, and the more popular a file, the higher its availability is. It’s perfect for large files that are most interesting when they’re fresh — in other words, it’s perfect for TV shows.

In many cases, I have been able to use BitTorrent to completely download a new TV show mere hours after the show airs. Like a TiVo user, I’m no longer bound to a specific time to watch my shows. I’m free to go out on Sunday night and still watch my show while it’s brand new. TV is now asynchronous.

Life is good.

But it could be better. RSS has revolutionized the way I consume some types of content, like weblogs and news. Instead of compulsively reloading a huge list of bookmarks looking for updates, my RSS reader scans them all every hour, and summarizes what’s new — allowing me to keep current on more sources with far less effort and waste.

Imagine if I had the same efficiency retrieving video files. With the addition of RSS, BitTorrent could really be taken to the next level, and I’d be able to forget about the plumbing of TV altogether. I want RSS feeds of BitTorrent files. A script would periodically check the feed for new items, and use them to start the download. Then, I could find a trusted publisher of an Alias RSS feed, and “subscribe” to all new episodes of the show, which would then start downloading automatically — like the “season pass” feature of the TiVo.

You know, for kids. Illegitimate uses of this system would obviously abound. But the potential legitimate uses are huge as well. For one, traditional content providers (like the TV networks) could take advantage of the demand for their programming by scooping the copyright infringers. If ABC released Alias on BitTorrent with advertising built in, the file could be delivered to their audience very fast, and would cost them next to nothing in distribution costs. The economics of producing video programming would be upended — each viewer of the program would, in effect, foot the bill for a tiny slice of the distribution overhead, causing a massive component of traditional media company infrastructure to become obsolete.

The point that Gillmor misses is that RSS files and JPEG graphics are small, and BitTorrent’s overhead makes it ill-suited for small files. The BT/RSS combo is potent, but its potential lies in using each technology’s strengths — RSS for syndication: low-overhead references to fresh content — and BitTorrent for the heavy lifting: swarming transport for high-demand, big files.

The beauty is that this synergy can so easily be realized. You can start publishing RSS feeds of BT files today: just point to a .torrent file in the element. A user who consumes the feed in their favorite RSS reader will be able to start the BitTorrent download with one click. Then, as the creators of RSS software catch on, they can automatically start the BT program when they encounter a new .torrent link.

The result: the TV distribution networks are completely end-run by an ad-hoc, decentralized, loosely-coupled network. And in the process, significant opportunities are afforded to independent content producers of audio and video to reach a mass audience with insignificant distribution costs.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.