How do I scrape a list of domains from a site?

SUBMITTED BY: Guest

DATE: June 15, 2019, 5:29 p.m.

FORMAT: Text only

SIZE: 2.8 kB

Raw Download

HITS: 409

Report
  1. How do I scrape a list of domains from a site?
  2. I am trying to compile a list of all of the domains that are on a site.
  4. ++++++++++++++++++++++++++++
  5. list of top cheapest hosting http://listtop.pw/
  7. Top 200 best traffic exchange sites http://listtop.pw/surf
  9. list of top gpt sites
  10. list of top ptc sites
  11. list of top ptp sites
  12. list of top crypto currency Wallets sites
  13. Listtop.pw
  14. Listtop.pw
  15. Listtop.pw
  16. +++++++++++++++++++++++++++++
  18. For example, I want to find all of the domains that are on Youtube.
  20. Would the search operation be site
  21. Youtube.com ".com" + ".net" + ".org" etc?
  23. Can that operator be tweaked so it includes all domains such as .io etc without having to mention it?
  25. Depending on what software you are using to scrape, you could use a regex for that.
  26. For example: /(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z0-9][a-z0-9-]{0,61}[a-z0-9]/g
  27. That would match the domain names. Example of that regex:
  29. Code:
  30. https://regexr.com/3au3g
  31. It will work on .io domains too.
  33. Edit: If you are talking about finding in google search, I do not think it will work. Not even the solution you are trying will work correctly.
  35. Depending on what software you are using to scrape, you could use a regex for that.
  36. For example: /(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z0-9][a-z0-9-]{0,61}[a-z0-9]/g
  37. That would match the domain names. Example of that regex:
  39. Code:
  40. https://regexr.com/3au3g
  41. It will work on .io domains too.
  43. Edit: If you are talking about finding in google search, I do not think it will work. Not even the solution you are trying will work correctly.
  44. Click to expand...
  45. What would be the best way to do so? Let's say the site is a directory. May or may not be that well indexed in Google.
  47. Home Page:https://rebrand.ly/Only-KEYWORD-Research-Needed
  48. STEP 1 :
  50. Plug the website on Ahrefs ( You can do trial = 7 USD for 7 days)
  51. STEP 2 : Click Linked Domain
  52. STEP 3: Result in less than 4 seconds
  54. What would be the best way to do so? Let's say the site is a directory. May or may not be that well indexed in Google.
  55. Topiano's idea is awesome to be frank. This will be the cheapest ( and the least time consuming) way to find the domains.
  56. Otherwise, I would simply make a python domain crawler.
  58. Python+ selenium can work perfectly.
  59. If the links are clickable you can use href property to get all the domains. If non clickable then get text from the body section and use regex to get all the links and use another regex to get domains only
  61. centexhosting.com
  62. hosting xs
  63. make money uploading videos
  64. w hostel boracay contact number
  65. hosting u bosni
  66. o domaine des delices marrakech
  67. netsukses.com
  68. sunbtc.space
  69. make money zaful
  70. undostreschocolate.wixsite.com
  71. olymptrade.com
  72. 2 host tick
  73. make money kindle publishing
  74. idomains reviews
comments powered by Disqus