In all my network related ruby projects I end up using this simple loop:
while buf = (socket.readpartial(1024) rescue nil) (data||="") << buf # process data here end
First let me explain why in general you want to use
readpartial instead of
read for network IO in ruby. I’m pretty sure people get confused over read/readpartial a lot because ruby’s
read doesn’t work the same as BSD-sockets’
read which you find everywhere in C code and other languages.
readpartial is actually much closer to the BSD-sockets’
read. When you call
socket.read(length) it will block until either the socket is closed from the other side or it has read
length bytes from the socket. In some cases this is ok, but a lot of the time it’s not what you want.
Take for example a protocol in which you receive instructions from a server. Each instruction can be from 10 to 100 bytes long and the instructions come approximately once a minute. In this scenario if you did
socket.read(100) you could potentially have to wait up to 10 minutes before that method returned and you could process the instruction(s).
readpartial to the rescue. It will return immediately with whatever data is available on the socket. If there is no data available it will wait until any amount of data becomes available and return it immediately. This means if a 10 bytes instruction came from the server and you did
socket.readpartial(100) you would get the 10 byte instruction back right away.
The one thing that kinda sucks about
readpartial is that instead of returning nil on an EOF it raises an EOFError. This doesn’t really help us create beautiful code. Thus the:
(sock.readpartial(1024) rescue nil) to catch the error and convert it into a nil. The problem with this is it will rescue any error not just EOFError, which is not optimal. I’ve often misspelled readpartial and the rescue nil will swallow the NameError, making it hard to figure out the problem. So I decided to do some refactoring and come up with a better solution for this loop I have to use all the time.
Here’s what I came up with:
class IO def while_reading(data = nil) while buf = readpartial_rescued(1024) data << buf if data yield buf if block_given? end data end private def readpartial_rescued(size) readpartial(size) rescue EOFError nil end end
Now I can write:
socket.while_reading(data = "") do # process data here end # the data var will still exist after we exit the loop's scope
This saves me a line, only rescues from EOFError, and still looks decent, probably even nicer than what I had before. Alternatively, if I don’t want it to automatically append the incoming data for me I can do:
socket.while_reading do |buf| # process each buf here end
Anyone have an idea for a better name than while_reading?
Also, just a note on when you should use
read instead of
readpartial. If you simply want to read all the data from a socket until it is closed then
socket.read will suit you fine. There are probably some other scenarios but I’ll leave those up to you to figure out.