Howto: Yahoo Pipes Auto-retweet recipe for Twitter
Tue, 02/03/2009 - 11:45 -- rprice
I have 5 or 6 twitter accounts, and some of them are only meant to serve as a discovery point for a blog, podcast, or community. That's why I cooked up this little pipe recipe to auto-retweet around two possible keywords, but NOT to retweet my own retweets if I did so from another account. Get it? Well, I'll explain:
Yahoo! Pipes is a free, web-based, visual programming environment. Those of you familiar with node-based editors like Shake or some other graphics and sound programs will feel right at home. Each "module" represents an input or output, or most importantly, ways to process and filter data as it travels through the pipeline. Pipes takes as input just about any feed or structured data - in this case, the river of tweets from a particular user's friends.
It's pretty easy to register for Pipes and create a new pipe. When you're done, you get a big blank piece of graph paper, and the list of sources, operators and other junk you can drop in on the left side.
Fetch Feed The first thing you need to do is go ahead and grab the twitter feed. In the left sidebar you want to find Sources > Fetch Feed and drag it on to the working area. In addition to the url to your friends list, you have to provide your username and password, because Twitter's API requires authentication.
Like this:
http://user:pass@twitter.com/statuses/friends_timeline.atom
At this point, if you take the output from the Fetch Feed module to the Pipe Output module, you have a working Pipe. But what you want to do here is filter out results based on certain keywords, so the output can be used to help you track a conversation.
Filter keyword 1 and yourself The next module you'll want to take a look at is under Operators > Filter. Drag this one out and take a look - this one reminds me of the Smart Mailbox rules in Mail.app, and it works in much the same way.
In this case we are going to:
Permit items that match all of the following Rules
item.title Contains fringe
item.title Does not contain bloggingfringe:
What does this mean?
The first rule just pulls out tweets that contain a single word or phrase you're looking for - in this case, "fringe".
The second rule filters out tweets by a particular user. In this case, the username contains the word you're filtering on, so you try to take that into account.
Again, you could connect the Fetch Feed to the Filter, and then to the Pipe Output and have a working version of the Pipe. In fact, I ran the pipe like this for a long time without anything else. But this is a two-keyword tutorial, and there are some other tricks I'd like to show you.
The next phase of your retweeting script is going to prepare the text of the tweets so they give a link back to the original author. You'll also remove a few precious characters by removing your own username from an @reply.
RSS and Atom documents coming out of your twitter friends feed give the title and body of each item the following format:
liberatr: @bloggingfringe I think it's really cool how you're retweeting our posts about fringe automatically!
This is the format you're going to pay attention to when you apply some simple regular expression replacement to the titles.
To give all of the retweets a more trational RT @user message format, you're going to grab the Operators > Regex module next. There are two rules you'll use:
In item.title replace : @bloggingfringe with :
In item.title replace ^ with RT @
Regex Rules
On the first rule, we are just cleaning things up in the case of a reply. Check i to ignore capitalization, so the rule matches @bloggingFringe or @BloggingFringe as well.
The second rule just adds a few letters to the beginning of the outgoing tweet.
Adding the second keyword
At any given time, there are other cultural events happening, not just fringe, so you might like to make two pipes, one pipe to track each event. If both keywords are going to the same account, you can use the Split and Union operators to apply different kinds of filters.
You need to use Split and Union in this case because your filtering has too many AND and OR questions:
If someone is tweeting about "fringe" AND it wasn't written by @bloggingfringe, OR
If someone is tweeting about "arts fest" OR "artsfest" AND not "fringe"
It's important to add the "and not fringe" to the end, because we don't want tweets appearing twice if someone mentions artsfest and fringe in the same tweet.
I'm going to gloss over a few steps here, because they are simple, and you've already done a few just like these: Filters for the second keyword
Create two new Filter modules
The first filter looks for your keywords, in my example the word could have spaces in it, so I needed a whole filter to spell out each variation
The second filter makes sure you're not duplicating tweets in the final pipe output
Put the Split operator at the very top, just after Fetch Feed. attach one of the outputs to the very first Filter and the second to the new keyword Filter.
Put the Union operator just above your Regex operator from before, with your two filtering decision trees going into the top, and the output from the Union going to the Regex, and that module's output the the Pipe Output.
Retweet Filter The last rule you're going to apply is one final Filter module. Since you're retweeting any tweet that mentions these keywords you care about, you may end up catching some retweets yourself, if your friends decide to share what you wrote. In the example, I have this filter appearing in between the Union and Regex modules, but you could also put this at the very top, above the Split.
Block items that match any of the following Rules
item.title Contains RT @bloggingfringe
item.title Contains Retweeting @bloggingfringe
Finishing Up
Once you get all your filters connected, splits and unions, regular expressions, feeds and outputs in order, you should have something that looks like this:
Pipe Recipe for Twitter: Auto-retweet
Please note the output (in the gray area at the bottom), and how it took the original tweet and just added 4 text characters to the beginning, adding a bit of context.
Now you have to make sure you Save your pipe, and finally click the link at the top saying Run Pipe... The only thing I want to warn you about is Publishing your pipe. Please remember that your twitter password is in there!
Pipes will format your output as RSS or JSON (or serialized PHP objects), as well as hooking into your favorite RSS aggregator and Yahoo! Alerts. As if that wasn't enough, think about all the RSS-to-Twitter applications that exist. My personal favorite is Twitterfeed.com - which is a free service, but I encourage you to donate a few bucks via PayPal, because it's just a fantastic thing to do!
Edit: also check out the excellent TweetBots, another way to get feeds to twitter, but it also has auto-follow and a cool way to allow multiple people to update the account using direct messages.
Gotcha: Make sure you select "Title only" in twitterfeed's interface, or the first few words of short tweets may be repeated!
I'm sure there are lots of other ways to filter your friends' tweets, and tons of ways to create a retweeting service, but this is one that works for me, and I just wanted to share with anyone looking for a little help.
Update: Double-retweeting
Since retweeting is so widespread, you may come across several instances where you are retweeting a retweet. Most times this will be pretty innocuous, but if you get into a situation where several of the people you're following retweet the same message, it's probably best to avoid those.
Example of the bad behavior:
RT @Obsessionful: I have been retweeted by @VociDance and @bloggingfringe. I'm awesome. xP: Obsessionful: I have.. http://tinyurl.com/bewhqw
RT @VociDance: RT @Obsessionful: "I think the thing getting me through this mountain of homework is the fact tha.. http://tinyurl.com/dx3x4g
RT @Obsessionful: I think the thing getting me through this mountain of homework is the fact that Arts Fest is t.. http://tinyurl.com/br65e5
No need to retweet it a second time.
Therefore, add one last Filter module just before the Pipe Output.
Double Retweet
Block items that match any of the following Rules
item.title Matches regex RT.*RT
That should take care of most double retweets.
After I first made this script, I was thinking about setting up two accounts that just retweet each other all day, and call them "echo" and "chamber"...
RT @echo RT @chamber RT @echo RT @chamber RT @echo RT @chamber RT @echo RT @chamber RT @echo RT @chamber RT @echo RT @chamber RT @echo RT...
You get the idea...
