Twitter Data: Mine, Export, Explore, Publish

Twitter Data: Mine, Export, Explore, Publish

Export your tweets to CSV, filter in a spreadsheet and publish.

Table of Contents

Edit

This content is open source. Help improve it.

At the start of 2018, I began retweeting all the most valuable information I found on twitter, as part of a learning process. Then, I would periodically scroll back and manually paste the links into the channels of a discord server, organized by category.

The problem with that, is if you are scrolling through your history, it has to load, and it’s easy to miss that particualr tweet where I know I left off.

Once I gave up scrolling my twitter history manually, I used a share button integration with discord, to directly send a given link to the proper channel.

Now I’m taking a break from twitter, transitioning from discord to spreadsheets, and in the process of consolidating a few years of link collecting on a variety of subjects. I have a lot to organize between 4 twitter accounts, a dozen discord servers where I collected\organized links, and a variety of other sources….

What do?

I started out using a web-app to grab my timeline, and then sort through them using CSV. More recently I got into python, where it’s pretty simple to build a script that will grab tweets based on a variety of queries.

For now this is just a collection of links from my research trying to figure out how to manage twitter in some sane way. Once I get a work-flow set up, I will share some of my own work here too.

Twitter History Export

  • Want to analyse your tweets? How to import Twitter JSON data exports into Excel - ZDNet

    It turns out that Twitter is using a non-standard version of JSON lists for its exports, and you’re going to need to make a quick change to the file it delivers if you want to be able to use your data in any analytical tooling. If you’ve been on Twitter as long as I have, that can mean editing at the very least a 100GB plus file (in my case two) to remove the text Twitter has added to the start of what would otherwise be a relatively normal list formatted set of JSON documents. You’ll need a decent text editor to make the changes. I recommend something like Visual Studio Code, which can happily load very large text files.

  • Show HN: Search your Twitter history with Algolia - Hacker News - Hacker News

    Instantly search across your entire Twitter history with a beautiful UI powered by Algolia.

Twitter API

Collections

Tweepy (python)

  • tweepy.api — Twitter API wrapper — tweepy 3.9.0

    This page contains some basic documentation for the Tweepy module.

  • Extended Tweets — tweepy 3.9.0 documentation

    When using extended mode, the text attribute of Status objects returned by tweepy.API methods is replaced by a full_text attribute, which contains the entire untruncated text of the Tweet. The truncated attribute of the Status object will be False, and the entities attribute will contain all entities. Additionally, the Status object will have a display_text_range attribute, an array of two Unicode code point indices, identifying the inclusive start and exclusive end of the displayable content of the Tweet.

Scripts

R Tweet

Apps

Re-Publishing Twitter Content

Nerd Stuff