You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »


default-scraper-config.json

{
  "magnetDL": {
    "name": "magnetDL",
    "url": "https://www.magnetdl.org/search/?q={{query}}&m=1&x=0&y=0",
    "list": "table[class=download] tbody tr",
    "result": {
      "name": [ "td[class=n]", "a[href]", "@title" ],
      "magnet": [ "td[class=m]", "a[href]", "@href" ],
      "size": "td:nth-child(6)",
      "seeds": "td[class=s]"
    }
  },
...


Testing 

Using jpillora/scraper https://github.com/jpillora/scraper/tree/master


Install scraper binary

curl https://i.jpillora.com/scraper | bash
sudo mv ./scraper /usr/local/bin


vi config.json

{
  "/search": {
    "url": "https://www.magnetdl.org/search/?q={{query}}&m=1&x=0&y=0",
    "list": "table[class=download] tbody tr",
    "result": {
      "name": [ "td[class=n]", "a[href]", "@title" ],
      "url": [ "td[class=m]", "a[href]", "@href" ],
      "seeders": "td[class=s]",
      "leachers": "td[class=l]",
      "size": "td:nth-of-type(6)"


    }
  }
}


Start Scrape Server

scraper config.json
2024/02/14 12:36:47 [scraper] Loaded endpoint: /search
2024/02/14 12:36:47 [scraper] Listening on 3000...


Attempt Scrape

curl "localhost:3000/search?query=halo"


Output

[
  {
    "leachers": "8",
    "name": "Halo.S02E02.XviD-AFG [TGx]",
    "seeders": "30",
    "size": "276.24 MB",
    "url": "magnet:?xt=urn:btih:xxx&dn=Halo.S02E02.XviD-AFG+%5BTGx%5D&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451&tr=udp://tracker.internetwarriors.net:1337&tr=udp%3A%2F%2Fexodus.desync.com%3A6969"
  },
...
]




  • No labels