default-scraper-config.json
{ "magnetDL": { "name": "magnetDL", "url": "https://www.magnetdl.org/search/?q={{query}}&m=1&x=0&y=0", "list": "table[class=download] tbody tr", "result": { "name": [ "td[class=n]", "a[href]", "@title" ], "magnet": [ "td[class=m]", "a[href]", "@href" ], "size": "td:nth-child(6)", "seeds": "td[class=s]" } }, ...
TestingÂ
Using jpillora/scraper https://github.com/jpillora/scraper/tree/master
Install scraper binary
curl https://i.jpillora.com/scraper | bash sudo mv ./scraper /usr/local/bin
vi config.json
{ "/search": { "url": "https://www.magnetdl.org/search/?q={{query}}&m=1&x=0&y=0", "list": "table[class=download] tbody tr", "result": { "name": [ "td[class=n]", "a[href]", "@title" ], "url": [ "td[class=m]", "a[href]", "@href" ], "seeders": "td[class=s]", "leachers": "td[class=l]", "size": "td:nth-of-type(6)" } } }
Start Scrape Server
scraper config.json 2024/02/14 12:36:47 [scraper] Loaded endpoint: /search 2024/02/14 12:36:47 [scraper] Listening on 3000...
Attempt Scrape
curl "localhost:3000/search?query=halo"
Output
[ { "leachers": "8", "name": "Halo.S02E02.XviD-AFG [TGx]", "seeders": "30", "size": "276.24 MB", "url": "magnet:?xt=urn:btih:xxx&dn=Halo.S02E02.XviD-AFG+%5BTGx%5D&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451&tr=udp://tracker.internetwarriors.net:1337&tr=udp%3A%2F%2Fexodus.desync.com%3A6969" }, ... ]