What's Next for AIart? (Dystopian Surprise!)

Lately, I've been a bit out of the loop with the rapid developments in generative AIart. Life threw a curveball with some eye issues and a corneal graft, which severely limited my computer time – even with NightCafe's regular emails urging me to claim my daily free credits.

My vision is improving, and I decided to jump back in by following one of NightCafe's invitations. This time, the free credits led to another invite to join one of their weekly challenges; they even pre-filled a prompt for me! I only caught the first couple of lines but decided to see what might happen. I'd previously experimented with prompts about AI/robots making art, so I wondered if that had "seeded" my new prompt, or perhaps another AI had "scanned" my earlier work on NightCafe (nothing impressive, just me playing around with the generative AI tech, by the way).

The image generated quickly, and I was genuinely taken aback by how dystopian and depressing it felt. It struck a chord, echoing my own concerns about the "look-at-me" culture of celebrity-seeking masses, which often feels like a desperate race to the bottom. Are we, as real artists, being replaced by soulless images churned out by "pseudo-intelligent" applications, all designed to grab attention so that advertisers pay more? Was I becoming just another cog in a process, simply there to click a link or hit enter?

"A melancholic robot with glowing eyes, standing in an abandoned art studio, surrounded by discarded paintbrushes and canvases, with a single tear of oil streaming down its metallic face, in the style of surrealism, with melting clocks and distorted perspectives, reminiscent of Salvador Dali, with a matte background and a somber, introspective mood."

Upon looking at the full prompt (above) and selected Flux model settings, I realised this prompt was likely just a refinement of my last work from a month or so ago (when I was effectively blind in one eye), rather than a new AI "spoon-feeding" me. The original prompt was: 

"a blind artist being assisted by an AI robot to create a large abstract painting surrealism Salvador Dali matte background melting oil on canvas."

I had used the Dreamscape XL Lightning model, a diffusion style model I was starting to like. However, the results weren't very Dalí-esque. A couple of the generated options had reasonable likenesses of Dalí, but nothing suggested he was blind or that a robot was assisting him. he just looked sad. Plus, the backgrounds were sharp and detailed, not melting, surreal or painterly. Was I to blame?, "Does my prompt need refining?" Yeah-Nah I just lost motivation and stopped.

Perhaps this is the true moral of the story for the artistic and creative communities: Keep an eye on what's going on, but don't worry too much. There's a lot of real life happening away from your computer or phone screen, and it's okay to let the over hyped generative AIart 'race to the bottom.' A bit sad really, if we want to benefit from the technology.

P.S. I even used Gemini Ai (Gemini 2.7 Flash) to help fix up my dictated jipperish for this post!

When "good enough" isn't really.

The general vibes I am getting back from my fairly intermittent and probably very personalized investigation of the massive deluge of commentary on "text to image" AIart is that of two extremes. Those vehemently against it (with the argument that it will be the end of artists, whatever that means) and those who see it as a wonderful new technology (bringing artistic creation to everyone). I don't agree with either. Like all technology there are pluses and minuses.

Created with Google
Deep Dream Generator

Let's begin by discussing a small personal project of my own. Can AIart tools be used to inspire my creativity? Specifically help me create a new cartoon character, I'm just using a simple prompt.

"cartoon of a watercolour artist painting with red hat"

I submitted this to a number of the more popular AIart tools. I definitely didn't expect similar results as I have been following most of the technical issues how each system has been created. The biggest difference is the data selected to train the neural networks, and remember there are usually two neural nets, one to review images (usually scrapped fairly randomly from the broader web, and the second to analyse associated text (usually within the websites or social media posts as an alternate text caption). Finally, it is very important to recognize that these systems are not copying the original images or even parts of those images. In the case of the image-based network it is just using "abstracted" dimensions (descriptors) that allow it to differentiate or group parts of the image. A previous post includes a video that explains these steps clearly. These could be shape, colour, texts, line ... I'll call them patches of style ... or perhaps how marks are shown might be a better description. Well, the results are a lot different.

Created in Nightcafe using a variety of methods but the same prompt.

First all these differences must make everyone realize that the common generic argument that these are copies or "rip offs" of an artist work, are largely uninformed ("fake new" in social media speak). Clearly, they are different because the neural nets were trained on different & very large datasets. In fact even submitting the same prompts can produce differences within a given tool. They are less copies and more just appropriation of some machine recognizable aspects of the style. You can quickly get into very muddy waters if you try to label this plagiarism, but even so I suspect5 legally this is a grey area.

The second myth to debunk is that because some of these systems are open source and free they are not promoted by greedy profit oriented vested interests (aka wannabe venture capitalist silicon valley types). Despite all this, yes all systems, restrict the number of free prompts you can submit and then offer ways to "buy" more "credits". To make it more game like you can "earn" credits sometimes by continuing to stay on the websites or use a specific part of their services. The really big costs in these systems are not really generating the work (although it might require more power/performance than the average PC or tablet today) it is the collating of the data and neural network training with millions of images and thousands on dimensions for characterization. These big projects requiring many people and seriously massive computer resources. Conveniently, a lot of this data is based on research projects, usually at universities and made available to the community relatively free of restrictions. I don't understand what is going on here, I do suspect some vested interests are deeply involved in academia, but I will restrain my suspicion until I know the work is noble and working towards a better future..

My final consideration for now is, are the photographers, illustrators and artists, whose work has been used in the training of these massive neural networks, happy about their work being included. I doubt they have even been consulted at all. Ok could they opt out? so that their style, characteristic mark making, colouring or composition doesn't keep turning up. I suspect not. The cat is already out of bag?

So, is AIart good enough (in terms of artistic quality)? Of course the quality of art is a very subject matter, and more relevant is the common perception that what is popular must be good. AIart has become very impressive in a short time but it doesn't live up to my expectations at the moment. There is always something a bit off (eg cropping, missing body parts, straying from the brief). Sure, you can refine the prompt, use the magic "words/modifiers", paint in or out, evolve and upscale, if you don't mind being kept using up your credits and buying more. Perhaps it is the public who have become more tolerant of lower resolution imagery and spend a lot less time actually looking at it. Instead, they are keen to scroll onto the next image, looking for that flashy colourful and more often moving picture. Well for a few seconds at least. The "good enough" threshold has perhaps become lower.

Can AIart be considered an artistic tool?

Despite the overwhelming impression you might get from just reading the latest social media post that there are two sides to the current #AIart cognitive dissonance.  The artist Luddites, who are claiming their creativityis being stolen versus the non-artistic utopians who think that text to image prompts lets them make great art in seconds. However, there is a lot of ground between the two camps.

I have worked with and spoken to other artists who have been very interested in what this new text to image tools, like Dall-E or Stable Diffusion can do. Many have been investigating these techniques as legitimate ways to create expressions of their unique creativity. In other words, is there life in these approaches as a tool. In my opinion it's not a simple yes/no answer but there is promise.

I have actually been interested and evaluating AI for a long time. I was even very early using the hashtag #AIart. In particular I have experimented with the idea of using such machine learning to capture the essence of my style of mark-making and “find” that within photos I have taken. This is one of the areas of what I have investigating under the “hand drawn photo”project. This method definitely has merit. Below is an example I created at nighcafe using their style transfer method. 

I starred with a handrawn image of an eagle's head I drew for Inktober. This was then used to “train” a simple neural net. I also supplied a photo, a self-portrait, which I had used as inspiration for myrevised profile on instagram. The training of the personalised network has to be scheduled on to suitable computer somewhere about on the internet and does take a while so the resulting image is not returned immediately. It is worth waiting for because now I have a black and white rendering. In my style of inkwork is closely follows the tones and shapes of the photo. I printed it on a laser printer and then hand-coloured it with my earth-coloured Ecoline Brush pens. I would consider this a work of my own art, but I have used some aspects of #Aiart in its creation.

#AIart and using Hashtags

I haven’t been as active on Instagram this past month, no real help from Alvin, just a lot going on. I did notice a massive drop in followers while I was off the net in the Kimberley's, and I have seen a more gradual return recently, and I probably attribute that to a bit more of a focus on using specific hashtags. The “fame” of a good image on Instagram is a fleeting thing, last a day or so at most. For am image to be found it has to be searched (either by your name, and not so many folk know me by the weird Instagram names I have, or via hashtags). Using hashtags is however a double edge sword. Some tags are so generic there are hundreds of competing images and often conflicting themes, for example #FollowTheSun is used a lot but more importantly it is used by the folks promoting bikinis and beach wear or travel services. My project of following the sun around Australia and my sketches don’t get notice beside a pretty girl or a dreamy location. Also the folk that follow the sun are more likely to be the fair weather followers (they want more and more beauty each day, if you don’t post it seems its the unfollow button for you).

A better strategy for me has been to focus down on more specific Hashtags. The good example here is the #AIart tag I started using it in Flickr & twitter late last year, and from about April this year on Instagram, to describe using neural networks trained to recognized a given style (or content) with a regular photo to produce a hybrid work somewhere between the photo and the art work. The real trick here is to get at least one of your images displayed in the opening nine. Instagram has a algorithm, no idea what it weights to include photos in this opening nine, but they probably represent the better and/or more popular images within this hashtag group. In the case of #AIart I am fortunate to currently have 5 of the top 9, within some 1410 post.  It is probably easy to have a hashtag only you use and you will be in that top 9 for sure, The magic happens when you encourage or inspire others to use the hashtags as well. Alternatively construct  composite tag so it shows up in more general searches (eg AI + art)  or the little related topics that often appears that the top of a search include it because it is similar. Now many more people might see your work.

Its not only Instagram posts that I tag with the same hashtags, I use them in twitter (where photos have a much shorter life, and without so many followers virtually no general exposure for me) but a hashtag means they can be found. (eg #janesweather, which means several have been on the TV weather report) I must admit I’m not a twitter fan, have had very little feedback/engagement and probably will abandon it soon. Finally tags, without the hash symbol have always been a good tool for finding things in flickr, they still are.

Keep on (hash) tagging. Just not with spray cans or a Posca on walls, landposts & postboxes.

Delving Deeper into the Worlds of AIart

 As the world around us changes rapidly, it's important to keep up with the latest technologies and trends. That's why I decided to delve deeper into the world of AIart, exploring the various offers and services available. Specifically, I've been experimenting with three of the current top AIart tools: Google Deep Dream Generator, Dall.E 2, and Stable Diffusion.

To test out these tools, I've been working with a local group of artists who meet fortnightly on Zoom to paint from a photo. Jeannine Desailly, a co-founder of our little Wednesday Wanderers painting group, kindly supplied a set of photos to paint. I will base my experiment on them.

My first step was to send a suitable prompt “pink lilly still life” to Dall.E 2. While the result was superficially impressive, there were a few things not quite right about the output. The image was overcropped and a little too zoomed in, it lacked the esthetics and composition of a still life subject.

But I'm not giving up on these AIart tools just yet. In my nextpost, I'll be exploring ways to "assist" these tools using these reference images. Stay tuned!

Exploring the Capabilities of Diffusion-Based AIart: OUTpainting

This will be the last post in my AIart series for now. I have been investigating the use of artificial intelligence (AI) to create stunning works of art. In particular, diffusion-based AI art has been gaining traction due to its ability to create realistic and artistic images.

Two additional features have became available under the diffusion process, INpainting and OUTpainting. INpainting involves removing a section of an image and replacing it with something that blends into the space. It is similar to context-sensitive replacement, as used in Adobe's software. On the other hand, OUTpainting involves adding extra areas outside the reference image in such a way that they seamlessly blend onto the image.

OUTpainting is particularly appealing to me here as it allows for the expansion of tightly cropped reference images. Expanding the image to a more classic still-life composition, playing with shadows and lighting to create a more realistic image. For instance, adding two full shadows, to give the image more depth and dimension.

I used Dall.E 2, at the openai website, as it is the most up-to-date version, which can add in extra objects to the image, such as a vase of flowers or a bowl of fruit etc, while still maintaining the original image's aesthetic.  Futher, Dall.E produces more photo-realistic images than some of the other diffusion-based AIart.

So my conclusion, text-to-image and particularly diffusion-based AIart is opening up new possibilities as a great tool/media for artists and enthusiasts alike. It's not a magic one-click to a masterpiece. It might be a shiny new toy for the wanna-bee celebrity artist, NFT worshippers or even those just wanting more likes on Instagram, The shine will soon wear off. By understanding and utilizing INpainting and OUTpainting features, we will all be able to create stunning and realistic images that were previously very difficult if not impossible to achieve. With technology continuing to advance, it will be interesting to see what other features will become available under the diffusion process or next AI advancement, and how they will be utilized in the world of art. I don't think the sky is falling in.

The Is it Art? Questioning AIart

The question everyone seems to be asking is AIart really Art?

View this post on Instagram

A post shared by Norm Hanson (@normhansonart)

My view is that it is just a different form of expression and it does have many of the characteristics of Art. It certainly will give more people the ability to create things. Something they probably lost as teenagers when they decided they couldn't draw. (Some of us continued to draw and maybe never grow up as suggested by Picasso) However, it is already being exploited by copycat and wannabe influencer types that are on that constant production of "content" treadmill for social media. When you first see an image created by DALL-E or Midjourney it's quite amazing but once you have seen a few the novelty does wear off, and with so many similar dystopian examples now being shown it's becoming a little disturbing.

Yet I see AIart in its many forms as a powerful tool in the hands of creatives. Something that is very clear in the above video, which is also worth watching as a simple explanation of what text prompted image generations actually is and the basics of how it works. So I'm keenly following the developments. On the upside, this technology can save a lot of time getting inspired and just fooling around with concepts. On the downside I do worry that the copycats will exploit true artists' ideas and styles, monetizing such works and thereby ripping off the less-discerning public. I can also see that many artists will not wish to have their works included any publicly available neural network, which inevitably means not having any of their art published on the net!. I don't have an answer to this. 

Such is (modern) life.

Opt-in & Opt-out Talk about #AIart and Have I Been Trained website

 Nothing brings the finer points of a debate into the spotlight than personal involvement.

The big AIart news of the last few days is that Stable AI will allow artist to opt out of being included in the next dataset being used to train the neural network to be used for Stable Diffusion 3. Well at least you can opt-out for the next couple of weeks, so follow this up now. 

There is a site, that can check whether you have been included in the massive Laion-5B & Laion-400m neural networks used in Stable Diffusion & Google's Imogen, yes they were trained on 5.8 billion images. I haven't established its legitimacy as an ethical stance, but it does seem legit. It doesn't appear to be a sneaky way to get more images (with so much hacking and phising establishing trust is a big issue in the #AIart discussions)

https://haveibeentrained.com/ 

However, when you see you are one of the artist whose images have been used to train the Lioan Neural Network and had no idea they are in there, some things suddenly confront you.

It took me very little time using the image search feature, to find two of my works. They were part of my Retracing Darwin exhibition in early 2010 and very early examples of my personal technique I call photoimpression. I actually don't mind if others study my method and even create examples of their own. I would like them to acknowledge me, which is becoming a hollow wish on today's web. I don't blame redbubble either, I am sure they didn't know and/or had not given permission either,

I really don't want my work, especially my own special techniques, style, mark-making colouring or composition used without my permission. This is the stuff that makes my work original. Firstly because I know I'll never be acknowledged, that others could profit from this work or contribution to this work to what is presented as their original, I also actually find a lot of the so-called art generated by these AI's a bit scary and I don't approve, and finally I find the whole process a bit morally questionable and not ethical.

So I've made up my mind, Now I do want to opt out.

Now I have to find out how? 

Damn! I have to do it image by image. Cest La Vie

#AIart Shoot Out

I was certainly an early follower of the developments in the deep dream style art filters, which use neural networks to recognize and amplify aspects of any photographic images. I must admit I like the potential of the forward seeking neural networks that can replicate line work and other mark making characteristics of the training image. I  chose to this called Deep Dream Style, and starting giving such creations the  hashtag #AIart (which seems to have caught on on Instagram). My prediction that these might become over used Art Filters is probably also an astute observation, there is certainly a plethora of such apps. Fortunately most are based around famous artist and great art, Even Adobe is joining the crowd offering researching a AI style filter that can “copy” the look from another photo. I still think these filters can be used creatively rather than as “look at me” plagiarism so obvious in social media today.

This photo was just an experiment, the first photo I took with my new HTC U Play mobile phone. I was trying out the AutoHDR feature and it handled the difficult strong lights and shadows well. Its just afternoon tea, aka Coffee & fresh figs. It is an interesting enough composition so I thought I would try the image across a variety of current #AIart tools (google deep dream style, on computer, Dremscopeapp & Prisma on phone)

Which rendering do you like best?

Second Generation selfportrait

 A lot of people on the Internet express the opinion that artist will go extinct because of #AIart. I am not one of them.

AIart Selfportarit of Artist on the Beach
(Style tranfer of a photo using a stable diffusion reference)

The current crop of text to image neural networks can be quite amazing, very good at capturing style, colouring and texture aspects of artistic mark making. Not so good at expressing an artist intent, specifically originality. Unfortunately, most of the current neural networks I’ve been trained up on rather depressing and dystopian art and images. I’m not sure that this is really the direction of art, more to do with the wasteland of trends on social media, where most images are found these days. Further so much depends on how you express yourself in the text prompt and modifiers. Will poets become the new visual artists?  Further it’s become common practice to keep refining the image and prompt. This is a lot like building up glazes or over-painting, creating layers of added texture and form and more eye appeal.

Already I can see that the simple first attempt text string output on social media from any of the AIart systems around at the moment, looks much the same, not quite right, unusually cropped and frequently scarily dystopian. Hopefully those jumping on the “one-click” filter style of social media influencer posts will soon become bored with the method. Leaving those interested in the potential for artistic expression time to experiment and refine methods. Thereby expanding the range of tools and media they have to hand. I for one look forward to this time.

Using a reference image within AIart I

Artificial intelligence (AI) has been making great strides in recent years, and one of its most fascinating applications is in the field of art. (AIart) With the help of generative adversarial networks (GANs), we are able to create art that is completely unique and unlike anything that has been seen before.

Most earlier neural networks used a general approach of starting with random noise. Early attempts, such as Portrait of Edmond de Belamy (2018), fetched a whopping US$ 350,000 at auction. However, these early attempts were crude and took numerous iterations. This approach uses two networks: the first is trained on images, called the generator, and the second scores how plausible the result of the iteration is, known as the discriminator. This approach is usually referred to as GAN (Generative Adversarial).

 The generator's job is to create images from random noise that look as if they could have been real. The discriminator's job is to differentiate between the generated images and the real ones. The two networks are trained together in a feedback loop, with the generator trying to improve its results and the discriminator trying to get better at identifying the generated images.

A slightly simpler approach to creating art with neural networks is to build a smaller network based on a single image or a small number of images from a specific artist. This approach is simpler than starting with random noise and can produce stunning results.

Back in 2015, German researchers from the University of Tubingen discovered that the earlier nodes of neural networks trained on images looked mainly at basic picture elements such as line types, shapes, directions, tones, and colour. These elements collectively made a good representation of "style". This means that by training a smaller neural network on a specific artist's style, we can use an existing image as the starting point to generate new artwork in the style of the artist works use.

Google's Deep Dream Generator was an early adopter of this approach and offered a Deep Style tab to perform this. Many other systems now exist that promise to interpret images, like your selfie, in the style of famous artists, for example.

In this example, I’m using a simple black and white pen sketch of my own, to control the line work but samples the original image colour. Its about what I expected and not exciting, I did however like the unexpected suggestion of a face showing up in the flower on the left.

Created with Google Deep Dream Generator

Style Transfer is available at Night Café, and in these example I used it for two different style reference images. The first, I'm starting with a different flower photo and again using my tree sketch, only this I'm using the option to make a short movie of the iteration involved in "finding" the resulting image.

In my last example I'm using a lesser-known Vincent Van Gogh's still life (why do so many start with Vincent’s work in  their exploration of AIart?)  

This time the results are stronger and closer to something that might be considered good more spontaneous art. But still not really impressive. 

In my next post I will look at trying to get a better colour likeness in the result.

Wandering into Midjourney on the way to Dall-E 2

I'm not sure how I managed to wander into the beta of Midjourney in a search to find Dall-E 2 (an AI based graphic tool that uses text to create an image, well so the claim goes!) I've registered an interest but not got into Dalle-E yet.

Ok what does midjourney make of the phrase "wandering in the light"?

These are just four alternatives  and I haven't, figured out how to refine one of them, yet. They do look like wandering, but out of the shadows rather than in the light. Amazing but still a little uncanny valley.

And here is a nice summary of how this form of AIart (text to Image) has been developing.

Will AIart replace artists any day soon? 

No I don't think so. 

Yet all the developments are occurring rapidly and amazing, and I'm guessing social media will soon be flooded with these unique-ish but not quiet original creations and like fake news how will we be able to tell the authentic,

Plein-air onto tablet and some pleasing #AIart

Yesterday I bit the bullet and tried out drawing my daily sketch directly on my HP spectre in Tablet mode, using the HP pen. There is a massive disincentive to working with the tablet, in fact anything with an LCD screen, in the outdoors, the screen is quite reflective and fine detail (and specifically small icons and text in the likes of Corel Painter are almost impossible to read) so sketching soon becomes a frustrating squinting, dabbing, missing, and confusion resulting in random lines all over the place. Thus I figured I’d start with a simple application, and the free version of Wacom’s Bamboo Paper fits the bill perfectly. There are hardly any on-screen control’s and they are big anyway. Further updates to the Microsoft ink workspace (that seemed to arrive in the Creator update to Windows 10) have improved the response of my pen and added a couple of levels of pressure sensitivity. There are only a pencil and highlighter style marker pen, with three width options in the free version of the bamboo paper app but that seems plenty to start with. Its also very natural to use. There is a colour change/select feature but accurately choosing colours in the glare didn’t appeal so I tended to pick the stronger colours and not worry too much..

I found a good spot, got out the tablet, shaded the screen with my body and made a start, timidly at first but I got brave and sketched lines and coloured over them with the highlight marker brush then back for more lines. Suddenly about 20minutes had slipped by and what I had done looked ok. I moved under the shade of a nearby tree and it looked better, ok better for my first outdoor attempt. It was after all just a field sketch (see above).

Today I stitched together 3 photos I took of table rock from the same vantage point and thought I should try improving the colouring (and tone of my sketch). It didn’t take long to realize this would be a perfect little test for the Neural Style feature of google deep dream generator. What I tried out was to use my bamboo paper sketch as the neural training image to filter my photographic panorama BUT to borrow the colour scheme from my photo rather than the sketch. In other words adopt the line work and flat highlighter style shading.from my sketch but stay close to the photographic colours (and tone). I must say I was impressed. I’m not sure how to take this AIart (Artificially Intelligent art) further but it does produce something with the sense of my touch and the way the place felt, something excitingly worthwhile.

Amazing but ... ... ...

It seems like only yesterday that I wandered in Mid Journey while looking to get an invite to DALL-E. It was actually almost a month ago but a lot has happened in the field of AIart, especially using artificially intelligent processes to create an image from a text prompt. (I prefer to consider these tools large scale and targeted neural networks, not intelligences)

Vermeer's  "Girl with a Pearl Earring" extended by DALL-E Outpainting

This illustration is from a PetaPixel article about a technique called outpainting,  were an image can be extended seamlessly, The approach uses shadows, reflections, and textures to extend the created background that is designed to blend perfectly with the original image, and uses its ability to pull in details from the millions of images run through its reference neural network. Not by finding actual object but shapes we see as real objects.

These technology advanced are truly amazing and wonderful tools for the creatives and original thinkers. Yet I am willing to bet that the technology will be dragged down on social to just create very dystopian, offensive, distastefully, look at me and wanna-bee influencer copycatism. They are already filling my Instagram feed. One bright outcome could be, it will probably lead to a big popularity to post AIart  as static images back on Instagram, making it largely a photo and interactive place again (#makeinstagramgreatagain) and not just a place to be force feed with ads, reels and the misguided opinions of the haters.

So what is my opinion, have we created ART?

No! The generation of this image by computer is amazing but it has taken a compellingly beautiful and intriguing work of art and made a mundane domestic scene with lots of clutter, you are unlikely to stop and wonder what she is looking at and just swipe past!

Using a reference image within AIart II

In the previous examples, I could see that the very basic style where being found but the Lillies had a wonderful colouring and I wanted to include that. So it was over to Google Image search looking for “pink flowers” “painting” and “Still life”. There were hundreds to search through but I soon found a link to a great watercolour demo by Barbara Fox. Not only do I have the "right colours" I also have the pigments used and a step-through of Barbara's painting processing. 

How would the Style Transfer AIart do?

Well wow, not bad. This is really generating a rich feeling for the subject, looking somewhat reminiscent of Norah Heysen's work.

My next post will look at a slightly different approach usually called Diffusion, that involves developing a latent third network.

PhotoProject :: My #HandDrawnPhoto approach to #AIart

The success of the Belemy portrait auction, has encouraged me to compare Obvious’s DAG approach to my use of Google Deep Dream algorithm. Deep dream uses a convolutional (foreword chaining) neural network to find and/or enhance patterns in an image. This can be applied in two ways Style & Inception (google’s terms). The inception process tries to find the characteristics of a given object/pattern within a photo and tends to produce surreal images. Style on the other hand  seeks to put characteristic patterns back into a photo based on features the trained neural network has recognized. This is the technique I have become very interested in as a way to put my characteristic marks (and sometimes colours) back into a photo. Hence the name Hand Drawn Photos for my approach. The method does have superficial similarities to Onvious’s DAG Approach.

Here I am using my sketch for inktober as the image to be processed and a photo of the same trees as my training image used to build the neural network (AI). The Photo here is acting like the discriminator in Obvious’s approach. The result (below) is tonally similar to the photo and kind of a messy squiggly version of something I might sketch.  But…

In this, my preferred approach, I am using the photo as the image to be modified and my sketch as the image to train the neural network and letting the system modify the photo to “find” my characteristic mark making (aka style) and modify the photo. In this case I am taking the colours from the photograph, but that is not essential.

I feel this second result is much cleaner and fresher, closer to something creative and artistic, Although I am strongly guiding the outcome by using my own sketches (as opposed to an automated algorithm). So the big question is “Do you consider this #HandDrawnPhoto to be legitimate AI art? I’m warming to answering yes.

If you are interested here is the link to my profile and public images on Google Deep Dream website. (Its free to join and use)

Love to hear your views,  please leave your comments here.

Using a reference image within AIart III

The world of image-to-text systems has been revolutionized by a new technique called Diffusion. Originally described as a Latent Diffusion Model, Diffusion has been widely adopted for its ability to encode training images into an encoded noise latent space, from which the system can then correctly decode the resultant images.

Night Café is offering a few techniques that take this a step further. These techniques have a diffusion-like ability, to begin with, a reference image with limited noise added, rather than starting with random noise. Night Café offers two diffusion approaches that start with a style image: Coherent and Stable Diffusion.

To understand the significance of this development, let's take a closer look at how Diffusion works. The core idea behind Diffusion is to train a neural network to predict the next pixel in an image, given the previous pixels. This process continues until the entire image has been generated. But unlike other image generation techniques, Diffusion is not limited to a specific set of images. Instead, it can generate an infinite number of images by starting with a random noise vector and decoding it into an image.

This is where Night Café's Coherent and Stable Diffusion techniques come in. These techniques allow the system to start with a style image, rather than random noise, and generate images that are coherent with that style.

Coherent Diffusion works by blending the style image with random noise and gradually removing the noise until the final image is generated. The result is an image that shares the same style as the reference image.

Not quite what I expected! Then I realize Norman Lindsay did paint a lot of scantily clad sirens!

Stable Diffusion, on the other hand, works by gradually adding noise to the style image until the final image is generated. This technique is particularly useful for generating images that are similar to the style image, but with slight variations.

In conclusion, Night Café's Coherent and Stable Diffusion techniques are a major breakthrough in the field of image-to-text systems. By allowing the system to start with a reference image, these techniques offer a new level of control and precision in image generation. The possibilities for creative applications are endless, and we can expect to see even more exciting developments in this field in the future. My next post will return to google's deep dream generator and its newest feature Text-2-Dream.

With thanks to Chat GPT which was able to translate my techno babble into easier-to-follow plain English (but I did have to correct it in a few place, so generative text AI is not perfect yet either). I left in its enthusiasm in the last paragraph even though I still have some reervations.

The Shape of Things to Come?

I’ve left to last, writing about aspects that I believe Ai techniques should be applied for photo editing and management in the future. The movement to apply different Ai based refinements into existing tools (eg masking, blending etc) will definitely continue. What I would love is to be able to train my own AI with my style of processing images and being able to supervise a more automatic “workflow” to suit both the photograph's content and my objectives.

Ted Forbes of the YouTube channel “The Art of Photography” has reviewed a new AI based photo editing offering called Imagen AI

This does sound a lot like something I have been looking for, a service that will automatically create edits for you using your styles (or generic “talent profiles”). At present, it appears very much aimed at bulk usage cases like event and wedding photographers and specifically from Lightroom. I hope the methods can be taken further, expanded to be more general and interfaced with other software/starting points.

So this is interesting, but not immediately helpful for me (whilst I do have an old version of Lightroom still but never use it) and I’m not holding my breath that it might become the next big thing.

Google have now also released its own text-to-image service also called Imagen, It's a competator for other similar AIart and whilst appropriate prompting can elicite photo-realistic images it's unlikey to be interest to or useful for most photographers.

Using a reference image within AIart IV

I’m returning to google’s Deep Dream Generator again and this time using the same Lilly photo as my reference and using their Text 2 Dream tab and the prompt “Watercolour painting in the style of John Singer Sargent.”

This one really hums.  The layout/composition has been altered a little, and the flowers are more Roses than Lillies BUT I can see it has added some butchered signatures, All is not perfect, YET!

My next post will return to Dall.E and investigate the feature of Outpainting.

The hyper-real, as a replacement for our photographs?

 

I was out scouting for my last photowalk, on a bright winters day with virtually no breeze. The Jells Park lake was forming a beautiful mirror with strong reflections. I am also testing out the new Skylum Neo extension for creating a multi-image stitched panorama, its nice and much faster than the stitching routine I'm currently using in ON1 Photo RAW.

Things are changing fast in generative AIart all the time and I was also testing out the latest version of Stable Diffusion (model SDXL 1.0) on Nightcafe studio and thought a sunny winter's scene like the above would make a good test. I used the text prompt -

"bright winter blue sky, some clouds over a still lake with strong reflection, yellow ochre grasses in foreground, Professional photography, natural lighting, shot on micro four third digital camera sharp focus"

With the above photo being given the defaults of a 50% Prompt Weight and 50% Noise Weight as the starting diffusion image for the generative process.

Certainly the capabilities of these generative AI systems has become really great, the detail is definitely starp and the reflections undoubted strong. However is it just a bit disturbingly too much on the hyper-real side?

"Hyper-real: More real than reality..." Brookes Jensen, Lenswork PodcastLW1362 - The End of the Trail, Sort Of