I for one welcome out GitHub Robot overlords; Embracing a robots.txt solution


Wired had a piece on GitHub robots and how the company doesn’t seem to be a fan. I was reading and reading to see what the reason was, and it came down too:

“I don’t want to see 100 of these tomorrow. That’s the fear. We don’t want to be the platform where you don’t want to run a project because you’re going to be accosted by so much of this activity.” — Brian Doll

That doesn’t seem to be too strong, and there is room for wiggle room. As we look at the “Internet of things” era, and how “social coding” opens so much up, especially with pull requests as a mechanism for systems to say “I think this may help, take it or leave it!”

ImageOptimiser is one good option. Imagine how many bits are wasted on the Internet due to sending large images? On my team I have an amazingly anal (in a good way) developer who wields ImageOptim and friends to great effect.

It is the tip of the iceberg on what can be done. The dream of automatic bug fixing, automatic performance optimizations, and friends are coming.

I understand why GitHub is wary, and how easy it can be too annoying.

What if we had a robots.txt for Git?

Instead of banning everything, how about coming up with a standard that would allow project owners to define how they want to deal with bots?

We could have a robots.txt^H^H^Hjson that declares how we feel about bots. It could contain info such as:

– bot user agent white list

– bot user agent black list

– limit which branches the bot should look at

If we embraced bots, and built support for them, what could we do?

– a way for bots to register for updates and be called back when changes are made (GitHub has good support for things like this)

– a way to separate pull requests that are from a bot, so the UI itself could show them off to the corner. This is a general feature that would be nice to be able to say “show me pull requests from a whitelist of users first”… priority inbox-esque

– Bots could ask GitHub itself which projects would like it to run (whitelist first) rather than scanning projects. Part of this would mean that the bot could say “show me projects that have images”, “show me projects that use Django”, etc.

What do you think? Should we embrace the bots? My answer is yes.

Leave a comment

Your email address will not be published. Required fields are marked *