Been asked a few times about how to get all a user’s tweets off twitter and onto a local file.

Activestate have a great python recipe to do it that works well. You just have to get python and BeautifulSoup onto your PC.

First install Python:

• Linux, it’s easy, just check your package manager and get python.
• Windows, it’s easy, go to activestates website a download and install ActivePython

Then install BeautifulSoup:

• Download BeautifulSoup tar
• Unzip it – you may need to untar it too. On Windows you can just use 7-zip or WinZip on the tar file as if it was a zip file. You will end up with a folder called BeautifulSoup-X.x.x.
• Open a terminal or Command Prompt and cd to the BeautifulSoup-X.x.x folder
• Type python setup.py install and hit return
• BeautifulSoup is installed

Now setup the program that gets the data and saves it for you.

• Open the activestate recipe page.
• Create a new text file in your Documents folder called as twitterArchive.py (make sure you get rid of the .txt extension)
• Edit the file and copy paste the recipe into the file:

  #!/usr/bin/python
  
  import time
  from urllib2 import urlopen
  from BeautifulSoup import BeautifulSoup
  
  # Replace USERNAME with your twitter username
  url = u'http://twitter.com/USERNAME?page=%s'
  tweets_file = open('tweets', 'w')
  
  for x in range(10*10000):
      f = urlopen(url % x)
      soup = BeautifulSoup(f.read())
      f.close()
      tweets = soup.findAll('span', {'class': 'entry-content'})
      if len(tweets) == 0:
          break
      [tweets_file.write(x.renderContents() + '\n') for x in tweets]
      # being nice to twitter's servers
      time.sleep(5)
  
  tweets_file.close()
  

• Update USERNAME as instructed in the recipe and save it
• Run the program. On Windows you can double-click it or run from a Command Prompt. On linux you’ll need to run it through python: python -c twitterArchive.py
• It will create a file called tweets in the same folder. After a while (it can take some time) the file will be filled with your tweets.