Is organic traffic from Google dead? Currently working on ProductSearch.pw to see if AI-generated sites are indeed killing hand-crafted niche websites in Google search results. It is an informational consumer goods website completely generated using AI. I am sharing the results over X, at the pinned post on my profile page. I will also be posting videos regularly for indie developers and solopreneurs interested in building in public. Follow me on X to be updated about the results of this experiment in real time.
Here's the tech stack I have been using:
12/1/2023 - This marks the end of my SEO experiment. I've learned that niche sites with valuable, user-focused content might be more effective than purely AI-generated ones. I think, going forward, I will avoid black or grey hat SEO methods as much as I can, focusing instead on creating genuinely useful content. I've shared the code I used for this 550-page ai-generated site on GitHub (link below), which includes various scripts for site functionality like generating breadcrumbs, child links, and user comments.
Besides this, I'm launching an indie gaming website. I've got multiple projects in the works, including an AI-powered news translation site in multiple languages. I'm also considering creating a course on Python or Hugo for YouTube and Udemy.
On a personal note, I'm working on enhancing my skills in JavaScript, React, and UX/UI design to improve my web design capabilities. Outside of these projects, I am continuing to teach university courses.
I'm excited to share my journey as a solo entrepreneur and indie hacker, balancing my academic career with my passion for web development and AI technologies. Thanks for following my updates, and I'll see you guys in the next video.
Get My Source Code Here: https://github.com/runeconnor/PySiteAI
11/11/2023 - Google Search Console is showing impressions, but no clicks as of this date. Only 1 page of the site has been indexed by Google, and the site is not really getting any traffic from Google. Next, I will be adding multi-language support featuring 20 different languages, which will skyrocket my page count from 550 to 11,000. I will also add 1 single consumer product to the site, 1 single page with 20 translations. The page will feature a Buy button, and an actual shopping cart, but "buying" the product will only send me an email notification, and not end in an actual sale. Hopefully I will be able to convince Google to index more of my pages by then.
9/29/2023 - It has been a wild few weeks, with vacations and other priorities getting in the way, but I have finally managed to finish the migration from Grav to a custom static site solution. I generate the pages for my website using a Python library that I created, and the site uses minimal PHP to bring everything together on-page. It is not multilingual like the Grav version, because I decided to focus on English-only pages at first. The site now has 550+ pages of 100% AI-generated content in English. The cost is $0.01 per page for the text, and even less for images.
1. Querying the OpenAI API using a single-threaded call was quite slow (3-5 mins per article
generated) - I wouldn't be able to scale as fast as I would like at the existing
pace.
Solution: Changed my code to support multi-threaded API calls, where I generate a
call every 1-2 seconds. Keep in mind the OpenAI API call limits when implementing a similar solution
yourself.
2. My CMS at the time (Grav) could not handle the 1.9k pages I added to the site, as page load
times skyrocketed. The entire folder structure is traversed by Grav, apparently, and it seems there
is a scaling problem. The devs of Grav are working on the scaling issue, but I wanted to go fast, so
I
decided to find another solution.
Solution: Uninstalled Grav, and started writing a
custom solution in pure PHP, all the while handling the migration from Grav using self-made Python
scripts. I was going to use another CMS like Hugo at first, but decided not to spend time on the
learning curve,
since this experiment is time-sensitive.
3. This is taking longer to implement than I initially anticipated.
Solution:
Honestly, I would have been better off just using Wordpress API. I will need to make my content
generation code work with Wordpress anyway, so it would have been smarter to focus on developing AI
content generation using Wordpress API instead of creating (almost) statically generated pages. Oh
well.
4. Getting gpt-3.5-turbo to output valid JSON consistently without dialing down the temperature really low is tricky. I implemented a two-step JSON decoding policy - I first try to decode the ai response using Python's standard json library, but that requires valid JSON. If the ai response is not valid, my code then tries the dirtyjson Python library in case the returned JSON is invalid. It works most of the time, but I still get exceptions from time to time, when the ai json response is truly invalid. This is with the ChatCompletion interface, of course.