creating a web crwaler using python libraries to search any keyword on google and getting the list of URL's and it's contents.
#for watching working video: https://drive.google.com/file/d/1669wblfE_Ud4raLUBviXeFUTTkkAHiRd/view?usp=sharing
Getting the data from the google by searching keyword.
prasing different types of data like:
- Title
- URL's
- Paragraph
- Heading
- Indexes
first crawling these data then connected our code to Database MySQL from the localhost and then storing the data in the format attached below
Our main objective was creating crawler and then cleaning of data Sub- objective was to connect to database and store the crawled data automatically
above both objectives were already completed.
After that we have to create a frontend for user experieence so user uses that for search the keyword and see the result on the frontend. Now I'm going to implement the semantic algorithm which will more personalize the search results using ML and NLP(next phase)