For my Bing Rewards Auto Searcher python script I was using a word list and randomly combining words to create phrases. This worked fine but I worried that the searches would be flagged by Bing because, well, they made no sense. So I looked for a way to easily gather a list of legitimate search phrases and there is no better place to get that than Google Trends. Google is nice and provides an RSS feed of their daily search trends so I went about creating a Google Trends keyword scraper that would parse that list and create a simple text file of each keyword that ends up trending. Below is the script.
Google Trends Keyword Scraper
The below script is pretty simple. Just copy and run, making sure that you create all the needed files and adjust the locations used below. You can do many things with this but I simply need it to create a txt file with a rolling list of the keywords.
import urllib.request import json import os def remove_duplicates(keywords): newlist =  for keyword in keywords: if keyword not in newlist: newlist.append(keyword) return newlist url = 'https://trends.google.com/trends/trendingsearches/daily/rss?geo=US' urllib.request.urlretrieve(url, '/root/python/keywords.xml') # exists f = open('/home/nginx/domains/myhelpfulguides.com/public/keywords.txt', 'r') old = f.read().splitlines() f.close() oldList =  for data in old: oldList.append(data) import xml.etree.ElementTree as ET tree = ET.parse('/root/python/keywords.xml') root = tree.getroot() freshList =  for title in root.iter('title'): freshList.append(title.text.lower()) freshList.remove('daily search trends') newList = oldList + freshList newList = remove_duplicates(newList) output = '\n'.join(newList) f = open('/home/nginx/domains/myhelpfulguides.com/public/keywords.txt', 'w+') f.write(output) f.close() os.remove('/root/python/keywords.xml')
I have made my list public so that anyone that wishes to use it can. You can check it out at https://www.myhelpfulguides.com/keywords.txt. The script that powers my list runs every 5 minutes so it should be very up to date.