Have You Been Scraped? Uncovering AI's Training Data

In the age of generative AIart and large language models, the question of concern for any creative artist (I am loath to call them "content creators" but social media does): 

Has our work been used to train AI without our knowledge or consent? A new tool offers some answers and a way to take action.

The website haveibeentrained.com allows users to search through vast, publicly researched AI training datasets like LAION 5B using text prompts. Curious about my own digital footprint, I decided to give it a try.

https://haveibeentrained.com/

Searching my name yielded numerous images from other Norm Hansons, but among them was a familiar face—my own. A self-portrait rock painting I'd posted long ago as a profile picture on Artists at Large had made its way into the dataset. While not overly distressed by this single instance, it did give me pause.

More concerning was the discovery that my charcoal sketch of Sir John Monash, created for an exhibition in 2018, had been scraped from my website. This unauthorized use of my work felt like a violation of my artistic rights.

Fortunately, the website offers a small measure of control. For individual images, users can tick a box that adds the image to a "Do Not Train" register, signaling to participating groups that you don't want your work included in future neural network training sets. For broader protection, entire domains can be registered.

It's worth noting that these actions are somewhat akin to closing the stable door after the horse has bolted. The data has already been used in training existing models. However, it's currently our best option for protecting our work moving forward.

This situation highlights a critical need for transparency and ethical behavior from those creating large language models, whether for legitimate research, commercial interests, or other purposes. As AI continues to evolve, so too must our understanding of its implications for creative rights and data privacy.

Have you checked if your work has been used in AI training datasets? Share your experiences and thoughts in the comments below.

Developing a New Self-Portrait for my Website

It was time to change my profile pic on my website my website. Not so much my profile self-portrait that’s the first image you see on my landing page. It’s traditionally been a self-portrait. The previous one was about my eye operation, an abandoned attempt to paint it and photograph the bandages.At the time, I was also slightly overwhelmed by the apparent significant progress in generative AI. Neural network learning applied to graphics and large language models, training on massive data sets scraped from the internet. In the end, I had combined a photo with my style of painting, using my own technique following what is usually called style AI.

My eye is on the mend and it’s time to update my portrait profile picture. Back in the beginning of the Covid lockdowns, I did a short month-long project painting various self-portraits in a variety of formats & styles. It was a lot of fun. One of the self portaits saw me holding up a small square canvas with a stylized version of my face in front of my face as I was photographed. The motivation was thinking about what we can believe as truthful on social media. It was a genuine photograph not photo-manipulated at all. Already people were suspicious that a lot of photos on social media claiming they were “photoshopped”.

Moving ahead a few years, we commonly have generative AI and deep fakes. In most cases, someone familiar with photography or art will still be able to spot inconsistencies usually to do with the lighting on the subject versus the background or perspective. But that’s another story. I was interested in revisiting the issue of what is real on the Internet and this time. I was definitely wanting to use AI, but style AI where I am using my own mark-making to modify my own photo. Just making the photo into line worked quite well. Perhaps disguising the impression of holding up something in front of my face so I simply added a semi-transparent square. 

To be honest the result hasn’t excited me but I have achieved what I was looking to do. I want to show I understand the potential of AI to make something original and creative, but not doing it in a copycat / "likeme" way.

PS: Should have taken my time before I uploaded the new photo. Did not really show what i wanted to say, so I went back to my original self portrait.

Where Do We Go From Here?

Testing Out New AI Tools for Writing - The Good 👍and The Bad 👎

I've been experimenting with some of the new large language AI models like ChatGPT Bard and Claude to help summarize and clean up my dyslexic writing. At first, it seemed amazing - I could just dictate my random thoughts and the AI would turn it into clear, readable text. I even had it generate content like blog posts, YouTube scripts, and Instagram captions.

However, I started noticing some issues:

• The AI can be overenthusiastic, especially when mentioning product names. It reads like advertising copy. I have had to rewrite these sections to keep them factual.

• Outrageous claims and incorrect facts. About 40% of the time, the AI includes claims or "facts" that are just plain wrong. I end up removing entire paragraphs.

• About 30% of the time, the content is good as is. The other 30% needs some reworking to tone down the language.

Clearly there's an issue here with misinformation. My current theory is that these large language models are trained on in-discriminant internet data containing conspiracy theories, misinformation, and bias. Garbage in, garbage out. 

I'm finding Anthropic's Claude model more reliable with fewer glaring errors. I have used it on this post, but I still have to carefully review any AI-generated text before publishing. 

As AI becomes more ubiquitous, it's crucial that we understand how these models are trained and what biases they may contain. We have to establish checks and balances, verifying information and not blindly trusting AI outputs.

I'll keep experimenting with AI writing assistants, AI in photography and digital Graphics (generative AI images such as the one abouve), but always maintain oversight. Stay tuned for more on responsible use of generative AI.

Asking AI for help writing this post

I had set up this topic to do a little test of AI chatbots from Open AI's (chatGPT) and Google’s (Bard), using the simple prompt “write a short blog post on why an old TV makes a great monitor for an artist's studio”. Well, it was a no contest, ChatGPT took old TV to mean a cathode ray screen and its text was overly enthusiastic and provided very dubious reasons. Bard did provide clearly correct information and a couple of items I’d overlook. Neither appealed for direct use as the "content" in this blog.

So then I tried Generative AI (Text-to-Image) comparing Stable Diffusion versus Dall.E (using nightcafe studio). I altered the prompt a little “Artist viewing a wall-mounted LCD TV to copy a reference photo and paint at easel”. Again pretty unusable perhaps except for the large one shown below. It’s superficially ok has some artistic merit and just might be partly on topic.

Worryingly he appears to be considering painting on the wall-mounted picture, which isn't a TV!
Using his right hand attached to his left arm!!

Maybe I’m getting a bit picky now. I’m a little spooked now that Google might be downgrading access to your blog if it’s using chatGPT. 

Co-incidence or Urban Myth?

"Google recognizes posts created from ChatGPT and downgrades them in search"

I've seen similar comments on several Instagram reels (I'd never followed them by the way they were pushed at me), google news, youtube and a couple of blog posts. Sounds like a bit of viral hype/myth to me but thought I would check my blog stats.

Well isn't that a coincidence?

I still doubt that things are moving that fast, and are so competitive. Maybe they are. There does seem to be a race to claim technical dominance of the ChatAI and AIart fields. Also I'm not sure I yet trust OpenAI to be driving the developments, there are the issues of secrecy, open letters by many important people and experts to have a pause in building of bigger models and not forgetting the whole issue of the ethics of scraping the uncensored net for data. Finally, it is logical that having the training of these deep networks based on these unverified sources include all the bias and unfettered untruths that are increasing within the unrestricted websites and social media, with many of those publishing "content" hiding behind anonymity, What should we expect?

Personally, I'm not thrilled that Google might be one potentially trying to dominate yet another field/service that so many people are flocking towards. Google has an extensive history of introducing new things and then killing them, They prefer to call it deprecation.

Anyway I'm suspicious it might be true and I rather let those interested find my thoughts rather than have to get through an unnecessary layer of possible google "censorship". So no more blog posts written (or even reviewed) by ChatGPT. I'll still be investigating how artists and photographers might use and be affected by this technology. 

So, if you are interested in my opinion please stick around.

The “A Picture is worth a 1000 Words” Project

This might be a bit long but it documents an interesting self-referencing project I envisioned to have Deep Learning AI create and reuse something in a self-referencing strange loop. I consider this a personal Art Project BUT it was interesting, and may I suggest worth reading. 

Does AI really know what it is doing?

 1. First up I gave chatGPT the following task.

It actually only wrote 499 words but they do seem intelligent and coherent enough (in a roundabout way). They are reproduced unedited below.

2. I took this text and duplicated it and literally added “two words.” then reproduced it on a single A4 sheet (I had to make it a landscape orientation and make the font tiny)

3.  I photographed it (to make it an image).

4. Then I submitted the same 1000-word quote as a prompt and a photograph of the text to Stable Diffusion as a reference/starter image. Not sure what it is trying to portray (if anything). Yet I can see how it is related to my 1000 word project.

The text from ChatGPT …..

The phrase "a picture is worth a thousand words" is a timeless adage that has been passed down through generations. It speaks to the idea that visual imagery can convey a wealth of meaning and emotion in a way that words cannot always capture. This quote has become so ubiquitous that it is often used in everyday conversation, advertising, and even in academic circles. However, the idea behind it is much deeper and more complex than the simple phrase suggests.

To truly understand the significance of this quote, we must first explore the relationship between words and images. Words are powerful tools for communication, but they have their limitations. They rely on the reader's ability to interpret them correctly, and they can be subject to different interpretations depending on the reader's perspective. Images, on the other hand, are more universally understood. They can communicate ideas and emotions in a way that is less dependent on interpretation and more dependent on the visual experience.

One of the main advantages of visual imagery is that it can capture complex emotions and ideas that are difficult to express in words. For example, an image of a sunset over a calm ocean can evoke a sense of peace and tranquility that is hard to capture in words. Similarly, an image of a child's face can convey a range of emotions from joy to sadness, without the need for a single word. These emotions and ideas can be conveyed in an instant through an image, while it would take much longer to express them through words.

Another advantage of visual imagery is that it can convey information quickly and efficiently. This is particularly true in the age of social media and the internet, where people are bombarded with information every day. A well-crafted image can capture a person's attention and convey a message much more effectively than a block of text. This is why social media platforms like Instagram and Pinterest have become so popular, as they allow people to share images and visual content in a way that is easily digestible and engaging.

Despite these advantages, it is important to remember that images and words are not mutually exclusive. In fact, they often work together to convey meaning and emotion. For example, a news article may use words to describe an event, but it is often accompanied by an image that helps to bring the story to life. Similarly, a marketing campaign may rely on both words and images to convey its message and connect with its audience.

In conclusion, the phrase "a picture is worth a thousand words" captures the idea that visual imagery can convey a wealth of meaning and emotion in a way that words cannot always capture. While words are powerful tools for communication, images have their advantages, particularly in their ability to capture complex emotions and ideas quickly and efficiently. Ultimately, it is the combination of words and images that can create a truly powerful and engaging message.

…..Actually only 499 words

I also submitted the prompt to Dall.E (although I couldn’t submit a reference/starter image). These don’t make a lot of sense to me. Although they are photorealistic?

And also a couple of other systems, similar to MidJourney, again not making a lot of sense.

But my favourite is still Style Transfer. I can see a little creative self-reference here, I’m just not sure it was created intentionally

All images, other than those created in Dall.E 4, where created at @nightcafe studio

PS: If you want to consider "worth" to be represented by bandwidth or file space required. Then an image will cost at least between 5 and 10 times that of the equivalent raw text, or even 100 to 300 times if you need something of higher resolution. SO a picture costs a lot more than 1000 words to move around the internet. 

Exploring the Capabilities of Diffusion-Based AIart: OUTpainting

This will be the last post in my AIart series for now. I have been investigating the use of artificial intelligence (AI) to create stunning works of art. In particular, diffusion-based AI art has been gaining traction due to its ability to create realistic and artistic images.

Two additional features have became available under the diffusion process, INpainting and OUTpainting. INpainting involves removing a section of an image and replacing it with something that blends into the space. It is similar to context-sensitive replacement, as used in Adobe's software. On the other hand, OUTpainting involves adding extra areas outside the reference image in such a way that they seamlessly blend onto the image.

OUTpainting is particularly appealing to me here as it allows for the expansion of tightly cropped reference images. Expanding the image to a more classic still-life composition, playing with shadows and lighting to create a more realistic image. For instance, adding two full shadows, to give the image more depth and dimension.

I used Dall.E 2, at the openai website, as it is the most up-to-date version, which can add in extra objects to the image, such as a vase of flowers or a bowl of fruit etc, while still maintaining the original image's aesthetic.  Futher, Dall.E produces more photo-realistic images than some of the other diffusion-based AIart.

So my conclusion, text-to-image and particularly diffusion-based AIart is opening up new possibilities as a great tool/media for artists and enthusiasts alike. It's not a magic one-click to a masterpiece. It might be a shiny new toy for the wanna-bee celebrity artist, NFT worshippers or even those just wanting more likes on Instagram, The shine will soon wear off. By understanding and utilizing INpainting and OUTpainting features, we will all be able to create stunning and realistic images that were previously very difficult if not impossible to achieve. With technology continuing to advance, it will be interesting to see what other features will become available under the diffusion process or next AI advancement, and how they will be utilized in the world of art. I don't think the sky is falling in.

Using a reference image within AIart IV

I’m returning to google’s Deep Dream Generator again and this time using the same Lilly photo as my reference and using their Text 2 Dream tab and the prompt “Watercolour painting in the style of John Singer Sargent.”

This one really hums.  The layout/composition has been altered a little, and the flowers are more Roses than Lillies BUT I can see it has added some butchered signatures, All is not perfect, YET!

My next post will return to Dall.E and investigate the feature of Outpainting.

Using a reference image within AIart III

The world of image-to-text systems has been revolutionized by a new technique called Diffusion. Originally described as a Latent Diffusion Model, Diffusion has been widely adopted for its ability to encode training images into an encoded noise latent space, from which the system can then correctly decode the resultant images.

Night Café is offering a few techniques that take this a step further. These techniques have a diffusion-like ability, to begin with, a reference image with limited noise added, rather than starting with random noise. Night Café offers two diffusion approaches that start with a style image: Coherent and Stable Diffusion.

To understand the significance of this development, let's take a closer look at how Diffusion works. The core idea behind Diffusion is to train a neural network to predict the next pixel in an image, given the previous pixels. This process continues until the entire image has been generated. But unlike other image generation techniques, Diffusion is not limited to a specific set of images. Instead, it can generate an infinite number of images by starting with a random noise vector and decoding it into an image.

This is where Night Café's Coherent and Stable Diffusion techniques come in. These techniques allow the system to start with a style image, rather than random noise, and generate images that are coherent with that style.

Coherent Diffusion works by blending the style image with random noise and gradually removing the noise until the final image is generated. The result is an image that shares the same style as the reference image.

Not quite what I expected! Then I realize Norman Lindsay did paint a lot of scantily clad sirens!

Stable Diffusion, on the other hand, works by gradually adding noise to the style image until the final image is generated. This technique is particularly useful for generating images that are similar to the style image, but with slight variations.

In conclusion, Night Café's Coherent and Stable Diffusion techniques are a major breakthrough in the field of image-to-text systems. By allowing the system to start with a reference image, these techniques offer a new level of control and precision in image generation. The possibilities for creative applications are endless, and we can expect to see even more exciting developments in this field in the future. My next post will return to google's deep dream generator and its newest feature Text-2-Dream.

With thanks to Chat GPT which was able to translate my techno babble into easier-to-follow plain English (but I did have to correct it in a few place, so generative text AI is not perfect yet either). I left in its enthusiasm in the last paragraph even though I still have some reervations.

Using a reference image within AIart I

Artificial intelligence (AI) has been making great strides in recent years, and one of its most fascinating applications is in the field of art. (AIart) With the help of generative adversarial networks (GANs), we are able to create art that is completely unique and unlike anything that has been seen before.

Most earlier neural networks used a general approach of starting with random noise. Early attempts, such as Portrait of Edmond de Belamy (2018), fetched a whopping US$ 350,000 at auction. However, these early attempts were crude and took numerous iterations. This approach uses two networks: the first is trained on images, called the generator, and the second scores how plausible the result of the iteration is, known as the discriminator. This approach is usually referred to as GAN (Generative Adversarial).

 The generator's job is to create images from random noise that look as if they could have been real. The discriminator's job is to differentiate between the generated images and the real ones. The two networks are trained together in a feedback loop, with the generator trying to improve its results and the discriminator trying to get better at identifying the generated images.

A slightly simpler approach to creating art with neural networks is to build a smaller network based on a single image or a small number of images from a specific artist. This approach is simpler than starting with random noise and can produce stunning results.

Back in 2015, German researchers from the University of Tubingen discovered that the earlier nodes of neural networks trained on images looked mainly at basic picture elements such as line types, shapes, directions, tones, and colour. These elements collectively made a good representation of "style". This means that by training a smaller neural network on a specific artist's style, we can use an existing image as the starting point to generate new artwork in the style of the artist works use.

Google's Deep Dream Generator was an early adopter of this approach and offered a Deep Style tab to perform this. Many other systems now exist that promise to interpret images, like your selfie, in the style of famous artists, for example.

In this example, I’m using a simple black and white pen sketch of my own, to control the line work but samples the original image colour. Its about what I expected and not exciting, I did however like the unexpected suggestion of a face showing up in the flower on the left.

Created with Google Deep Dream Generator

Style Transfer is available at Night Café, and in these example I used it for two different style reference images. The first, I'm starting with a different flower photo and again using my tree sketch, only this I'm using the option to make a short movie of the iteration involved in "finding" the resulting image.

In my last example I'm using a lesser-known Vincent Van Gogh's still life (why do so many start with Vincent’s work in  their exploration of AIart?)  

This time the results are stronger and closer to something that might be considered good more spontaneous art. But still not really impressive. 

In my next post I will look at trying to get a better colour likeness in the result.

Delving Deeper into the Worlds of AIart

 As the world around us changes rapidly, it's important to keep up with the latest technologies and trends. That's why I decided to delve deeper into the world of AIart, exploring the various offers and services available. Specifically, I've been experimenting with three of the current top AIart tools: Google Deep Dream Generator, Dall.E 2, and Stable Diffusion.

To test out these tools, I've been working with a local group of artists who meet fortnightly on Zoom to paint from a photo. Jeannine Desailly, a co-founder of our little Wednesday Wanderers painting group, kindly supplied a set of photos to paint. I will base my experiment on them.

My first step was to send a suitable prompt “pink lilly still life” to Dall.E 2. While the result was superficially impressive, there were a few things not quite right about the output. The image was overcropped and a little too zoomed in, it lacked the esthetics and composition of a still life subject.

But I'm not giving up on these AIart tools just yet. In my nextpost, I'll be exploring ways to "assist" these tools using these reference images. Stay tuned!

Autodraw, another AI assisted drawing tool

I am late discovering this experimental tool from google labs, called autodraw.com. It dates back to mid 2017 and uses the same technology to guess an object as google's quickdraw. Only this time it lets you sketch out the object you want and "It pairs machine learning with drawings from talented artists to help everyone create anything visual, fast." 

Its as simple as that. The graphics are good (created by graphic artists), with Creative Commons Licensing, so free for non-commercial use, easy to download and use. Its also a bit of fun  although you may find it quickly reaches its limits of what it might recognize.