Meandering Out Loud

Musing, Muttering, and Mischief Along a Random Path

Collecting Senders From Gmail Using POP3

Over the years, we’ve had a lot of people submit jokes for LOL. Since we’ve recently upgraded the site to allow user submissions, we thought it would be nice to let them know they can now add their own jokes (Truth be told, we were slow adding them).

I’m not going to go into great detail, but I thought I’d post a mini howto. Note: I’m not supporting this code, but you are free to use it

Firstly, Ruby 1.8’s POP3 implementation doesn’t support SSL. stunnel provides encrypted channels for software that doesn’t understand SSL. I run cygwin on my Windows box to get access to essential tools. Note: Configuring cygwin and installing it is beyond the scope of this entry, so ask Google for help.

A quick search for gmail and stunnel allowed me to cobble together the following configuration file called gmail-tunnel.txt

client = yes
debug = debug
foreground = yes

[pop3s]
accept = 127.0.0.1:42
connect = pop.gmail.com:995

Open up a command window and issue this command to start the tunnel:

C>stunnel gmail-tunnel.txt

Save the following code in rpopget.rb and then execute it from another command window while the tunnel is active. It will write a file called address.csv that lists each address and the number of times that person sent mail.

#!/usr/bin/ruby

require 'net/pop'

HOST = 'localhost'
USER = 'YOURACCOUT@gmail.com'
PASS = 'YOURPASSWORD'

$addr = {}

def add_address(a)
    $addr[a] = $addr[a].nil? ? 1 : $addr[a] + 1
end

Net::POP3.start(HOST, 42 , USER, PASS) do |pop|
    if pop.mails.empty?
        puts 'No mail'
    else
        pop.mails.each do |email|
            $stderr.printf(".")

            lines = email.header.split("rn")
            lines.each do |l|
                if l =~ /^From:/
                    if l =~ /^.*/
                        add_address($1)
                    elsif l =~ /^From: (.*@.*)[ ]*/
                        add_address($1)
                    end
                end
            end
        end
    end
end

open("address.csv", "w") do |f|
    f.puts "Address,Count"
    $addr.each_pair do |a, c|
        f.puts "#{a},#{c}"
    end
end