Table of Contents
Introduction
To say that AI has become an important part of many businesses is an understatement. Google and Bing both have AI bots built into their results. Many customer support chatbots have been upgraded or replaced by these new tools. We’ve only begun to scratch the surface of what these tools are capable of. Much of the talk has been around the big picture of AI doing everything. I wanted to take a smaller view of these AI tools and see how I can use them in my day-to-day work.
Those of you not familiar with my background may not realize I’m not a trained writer. My background is a 3D artist and game developer. I have no training in writing, especially not of any technical means. My asset is my industry knowledge, while my weakness is writing. I’ve been working on this, but now with AI promising to be able to output an entire article, I wanted to see if it would work for us.
This is not an exhaustive research into all AI tools. Instead, this is a quick overview to see if it is something I can incorporate into my work and to begin exploring how Puget will approach AI when it comes to writing. This is a rapidly evolving field, so everything discussed may change in a few months time.
Initial steps
While writing my latest blog post about CPU and GPU Rendering, I decided to see what AI was capable of. Using the free version of ChatGPT (3.5), I went in blind, as any new user would. I simply asked it, “Write an article in the style of Puget Systems, explaining the pros and cons discussed in the previous response, and give examples of where CPU and GPU are each most effective.” What popped out was… ok, I guess. It is not something I would be comfortable publishing on our website.
Image
Issues
ChatGPT seems to like to make things into lists. This does make some sense as it was trained on large swaths of content, and much of what is on the internet, particularly on sites with lots of traffic, are lists. This is a quick and easy way to write, so many people wrote this way to get lots of content out as fast as they could. It is also an easy article format to read when trying to get the key piece of information quickly. That is not to say you can’t get a standard article out of ChatGPT, but depending on if you are trying to compare two things or do a pros and cons article, it heavily leans into the list format.
Another problem is with the data the AI model contains. Often this data is out of date or doesn’t have much context behind the data. For example, in trying ChatGPT for the rendering article, I asked it to provide some examples of popular renders for CPU and GPU, and the first engine it cited for CPU rendering was Maxwell Render.
I felt this was an odd inclusion for a few reasons beyond the fact that Maxwell isn’t widely used. First, while yes, it can use the CPU for rendering, they added GPU rendering support back in 2016, so it is clearly not only a “CPU Rendering Software.” It’s hard to fault the AI too much, as all the other examples for CPU rendering (Arnold, V-Ray, LuxCore) also have GPU options now. They could have included Corona from Chaos, as it is a very popular CPU only renderer.
Second, Maxwell hasn’t had an official update in over 4 years, and very little communication from the developers in that time. It doesn’t even have official support for NVIDIA’s 4000 series GPUs. Instead, users need to manually modify several of the program’s files to get them to work. This is the sort of detail that isn’t talked about in many articles, so the AI seems to have missed it.
The last problem is a little harder to define. The issue is that it just doesn’t sound like me, or Puget Systems in general. Every author has their own voice. For our hardware review articles, Matt, Evan, and I try to align our writing so that all the articles released for a specific piece of hardware feel unified. Because these LLMs were trained on such a wide range of authors and topics, the articles they put out do sound a bit generic. This may or may not work for certain people or use cases. There is already a lot of “business speak” to the way much content is written, but for a blog post, it can sound dry and lifeless.
Prompt Engineering
The common narrative online is that to get the most out of these models, you need to learn how to direct the AI to do what you want. There is no shortage of suggestions on Twitter (or X) or blogs. Not all are helpful. Some were useful at one point, but then as the models have evolved, have become less so.
I’ve seen it suggested several times to include a sentence along the lines of, “You have 30 years of experience as a {insert profession}”. The intention here is to push the AI to include more detailed information and not just surface level information. However in my testing, it didn’t really provide much more information. I don’t know if this is because the topic at hand, CPU vs GPU rendering, is largely covered by highly knowledgeable people, or that its not covered that often, so the pool of information is not extensive, but detailed.
One that I’ve really liked is “There is incorrect information in this article. What is it?” This returned additional information that the initial prompt did not include. Being able to use this specific prompt to double check for incorrect or missing information is extremely useful, even for people that did not use AI for the initial writing.
Generating Data vs.Repeating Data
One difficulty for us at Puget using AI is that we are the ones generating data. We are creating new information, and then trying to present that information in a way that people find helpful. AI excels at reusing existing data that is already online. Trying to merge the two has been difficult as AI tends to make up data if there isn’t enough. We could even give it the exact results, but because it has never heard of this new GPU, it will often make odd comparisons. Or worse, state facts that are untrue. That is a no-go for us. Our early tests with this often caused us to spend just as much time fact-checking and correcting the text that we could have written it from scratch in the same amount of time.
Beyond the accuracy, which will be overcome in time, is the security of this new information. If we are testing out a new video card that hasn’t been announced, will any date we input find its way into the model’s database and then into the public? There have been several instances of companies’ data being leaked by employees using ChatGPT. Often we are dealing with a few days between when we perform our testing and the articles go up, so it seems unlikely that it would be found that fast, but stranger things have happened.
What was helpful
So far, this sounds rather negative. However, I actually feel like AI is very valuable to someone like me who isn’t the strongest writer. Probably the most helpful thing is getting past the dreaded blank page. Many people who write will tell you one of the most difficult things is just getting something down on the page. Staring at a blank page can be very intimidating, so much so that a quick Google search will return thousands of suggestions on how to get started. With AI, you can give it the premise of your article, and it will return a fairly usable article. Some may find it much easier to start with something that needs corrections and to fill some missing gaps than it is to start from scratch.
This is what I did with the rendering article. What it gave me was okay, but there were a lot of missing details and nuances that I was able to add. I also didn’t like the formatting and felt some information was repeated, so I was able to rearrange sections, add what needed to be added, and rephrase some things to sound more like me.
Image
Another feature that I’ve found useful, for anyone that does writing is to give it a paragraph or two, and as for a few variations. This will give you a few different ways of saying the same thing. It is easy to get stuck in a rut in writing and become somewhat monotone. Having a tool that can quickly give you a few different options helps to improve the readability of anything you are writing, making you a better writer. I used this in our recent round of CPU testing, where I had to write multiple articles about rendering, all of which were essentially the same. Just reusing the same text multiple times is not great for SEO or people who want to read each article.
We are also seeing more and more tools using AI to assist in writing. Grammarly introduced an update that is very useful in improving the quality of your writing. Hemingway Editor will analyze the text and provide feedback on the readability. The list goes on and on. Even for people who aren’t going to use AI to generate the text, these AI tools will be very useful to improve your writing.
Ethics
I would be remiss to talk about AI without addressing the ethics concerns. Many of these models, especially the early ones, were trained on a variety of texts, some are under copyright, and some are not. Numerous people objected to the use of their text being used without permission. This has led to a very murky gray area that will likely lead to many court cases. These legal questions are extremely important, and safeguards do need to be put in place. I do not have enough information personally to tell you what models are more or less likely to include copyrighted material. For a large part, this will be a moving target anyway, as the AI companies take steps to remove offending content from their platform.
This also leads to the issue of people who actively want to be included in AI models but want to be attributed. Here at Puget Systems, for example, because we generate and publish the data for free anyway, we would like to be included when someone asks, “What is the best computer for game development,” but as a company that also sells those computers, we want people to know who created that information and then come back to us. Microsoft’s Bing Copilot, for example, returns the typical breakdown of computer hardware but also cites us as the resource for that information and provides a link to us for more information. This makes sense for a search engine (ironically, Google’s Bard does not provide links), but how other models handle that will be important to address.
Custom model
The next step for us would be to customize our own model and run it on our own server. This is something that Dr. Don Kinghorn has been exploring and writing about. Theoretically, we could have a model trained on the hundreds of articles we have already produced. This would give us the ability to have the model create something that sounds like us. It could potentially be able to read our database of test results to accurately compare hardware without making things up.
What this would cost to set up and maintain is beyond my knowledge, but that is the exact thing many companies are looking into right now. Every company has generated a large amount of data, and sometimes it can be difficult to access or parse through that data. Companies are looking for custom AI models trained on their own data and running on their own hardware to address the issues discussed above.
Conclusion
My dabble with AI, especially through platforms like ChatGPT, has been an interesting experience. From churning out initial drafts to suggesting different ways to phrase things, AI has been a bit of a mixed bag—sometimes helpful, sometimes not so much. But issues like relying on outdated data or struggling to maintain my own unique voice in the midst of generic outputs have given me pause.
These tools are evolving rapidly. New models and integrations appear each week. In the process of writing this article, several new tools have been released. Microsoft has integrated Copilot directly into Windows.. I suspect the issues described above won’t be much of an issue for much longer.
Looking forward, the idea of tailoring AI models to suit our specific needs is intriguing. Imagine having a custom AI buddy who knows our articles inside out, and can then offer recommendations based on your specific configuration. So, while AI promises to make writing easier, it is not at the point of writing full articles or providing industry analysis without significant human input.