Been asked a few times about how to get all a user’s tweets off twitter and onto a local file.
Activestate have a great python recipe to do it that works well. You just have to get python and BeautifulSoup onto your PC.
First install Python:
- Linux, it’s easy, just check your package manager and get python.
- Windows, it’s easy, go to activestates website a download and install ActivePython
Then install BeautifulSoup:
- Download BeautifulSoup tar
- Unzip it – you may need to untar it too. On Windows you can just use 7-zip or WinZip on the tar file as if it was a zip file. You will end up with a folder called BeautifulSoup-X.x.x.
- Open a terminal or Command Prompt and cd to the BeautifulSoup-X.x.x folder
- Type
python setup.py install
and hit return - BeautifulSoup is installed
Now setup the program that gets the data and saves it for you.
- Open the activestate recipe page.
- Create a new text file in your Documents folder called as twitterArchive.py (make sure you get rid of the .txt extension)
- Edit the file and copy paste the recipe into the file:
#!/usr/bin/python import time from urllib2 import urlopen from BeautifulSoup import BeautifulSoup # Replace USERNAME with your twitter username url = u'http://twitter.com/USERNAME?page=%s' tweets_file = open('tweets', 'w') for x in range(10*10000): f = urlopen(url % x) soup = BeautifulSoup(f.read()) f.close() tweets = soup.findAll('span', {'class': 'entry-content'}) if len(tweets) == 0: break [tweets_file.write(x.renderContents() + '\n') for x in tweets] # being nice to twitter's servers time.sleep(5) tweets_file.close()
- Update USERNAME as instructed in the recipe and save it
- Run the program. On Windows you can double-click it or run from a Command Prompt. On linux you’ll need to run it through python:
python -c twitterArchive.py
- It will create a file called tweets in the same folder. After a while (it can take some time) the file will be filled with your tweets.