Replies: 2 comments 1 reply
|
I'm not sure what's inside your export const config = {
url: "https://www.example.com",
};
export default function({ doc }) {
return doc.html().replace(/Example Domain/g, 'HELLO WORLD');
}Output: [
{
"url": "https://www.example.com",
"data": "<!DOCTYPE html><html><head>\n <title>HELLO WORLD</title>\n\n <meta charset=\"utf-8\"/>\n <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\"/>\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\"/>\n <style type=\"text/css\">\n body {\n background-color: #f0f0f2;\n margin: 0;\n padding: 0;\n font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n \n }\n div {\n width: 600px;\n margin: 5em auto;\n padding: 2em;\n background-color: #fdfdff;\n border-radius: 0.5em;\n box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n }\n a:link, a:visited {\n color: #38488f;\n text-decoration: none;\n }\n @media (max-width: 700px) {\n div {\n margin: 0 auto;\n width: auto;\n }\n }\n </style> \n</head>\n\n<body>\n<div>\n <h1>HELLO WORLD</h1>\n <p>This domain is for use in illustrative examples in documents. You may use this\n domain in literature without prior coordination or asking for permission.</p>\n <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n\n\n</body></html>",
"timestamp": "2024-02-24T14:20:43.574054+01:00"
}
] |
A little insight: In rare cases, the full JSON with all the data a website uses is in the |
|
If you're looking to scrape specific HTML elements while discarding others based on a regex pattern, you can achieve this by first retrieving the raw HTML content and then applying your regex pattern to filter out unwanted elements. Here's an example of how you might approach it: `// Get the raw HTML content of the body element // Apply regex pattern to filter out unwanted elements // Now 'filteredHtml' contains only text content, without any HTML tags |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I have been using element.html() to see if I get something back at all. But now, in one case, I want to use the actual raw HTML to scrape data. But I can't parse it. For example:
body.html().replace(/REGEX/g, '')doesn't work. The reason I can't use element.text().replace() is because I want the actual HTML with elements but discard all others using regex.Is there some magic in Flyscrape that outputs raw HTML? I have also tried { parse }, but can't make sense of it.
EDIT: I discovered .filter, but it doesn't behave the same as .map. Waiting for #49
All reactions