Python How to get all the links from a website

SUBMITTED BY: alemotta

DATE: April 1, 2017, 11:11 p.m.

FORMAT: Text only

SIZE: 448 Bytes

Raw Download

HITS: 1193

Report
  1. Python How to get all the links from a website
  3. This example will get all the links from any websites HTML code.
  5. To find all the links, we will in this example use the urllib2 module together
  6. with the re.module
  8. import urllib2
  9. import re
  11. #connect to a URL
  12. website = urllib2.urlopen(url)
  14. #read html code
  15. html = website.read()
  17. #use re.findall to get all the links
  18. links = re.findall('"((http|ftp)s?://.*?)"', html)
  20. print links
comments powered by Disqus