Ryan Mitchell - Separating Bots from the Humans - a podcast by DEF CON

Materials Available here:https://media.defcon.org/DEF%20CON%2023/DEF%20CON%2023%20presentations/DEFCON-23-Ryan-Mitchell-Separating-Bots-from-Humans.pdf

Separating Bots from the Humans

Ryan Mitchell Software Engineer, LinkeDrive Inc

There’s an escalating arms race between bots and the people who protect sites from them. Bots, or web scrapers, can be used to gather valuable data, probe large collections of sites for vulnerabilities, exploit found weaknesses, and are often unfazed by traditional solutions like robots.txt files, Ajax loading, and even CAPTCHAs. I’ll give an overview of both sides of the battle and explain what what really separates the bots from the humans. I’ll also demonstrate and easy new tool that can be used to crack CAPTCHAs with high rates of success, some creative approaches to honeypots, and demonstrate how to scrape many “bot-proof” sites.

Ryan Mitchell is Software Engineer at LinkeDrive in Boston, where she develops their API and data analysis tools. She is a graduate of Olin College of Engineering, and is a masters degree student at Harvard University School of Extension Studies. Prior to joining LinkeDrive, she was a Software Engineer building web scrapers and bots at Abine Inc, and regularly does freelance work, building web scrapers for clients, primarily in the financial and retail industries.

Ryan is also the author of two books: “Instant Web Scraping with Java” (Packt Publishing, 2013) and “Web Scraping with Python” (O’Reilly Media, 2015)

Twitter: @Kludgist

Amazon Author Page: http://www.amazon.com/Ryan-Mitchell/e/B00MQI8TVQ

Website: http://ryanemitchell.com

Further episodes of DEF CON 23 [Audio] Speeches from the Hacker Convention

Further podcasts by DEF CON

Website of DEF CON