Using a reference image within AIart I

Artificial intelligence (AI) has been making great strides in recent years, and one of its most fascinating applications is in the field of art. (AIart) With the help of generative adversarial networks (GANs), we are able to create art that is completely unique and unlike anything that has been seen before.

Most earlier neural networks used a general approach of starting with random noise. Early attempts, such as Portrait of Edmond de Belamy (2018), fetched a whopping US$ 350,000 at auction. However, these early attempts were crude and took numerous iterations. This approach uses two networks: the first is trained on images, called the generator, and the second scores how plausible the result of the iteration is, known as the discriminator. This approach is usually referred to as GAN (Generative Adversarial).

 The generator's job is to create images from random noise that look as if they could have been real. The discriminator's job is to differentiate between the generated images and the real ones. The two networks are trained together in a feedback loop, with the generator trying to improve its results and the discriminator trying to get better at identifying the generated images.

A slightly simpler approach to creating art with neural networks is to build a smaller network based on a single image or a small number of images from a specific artist. This approach is simpler than starting with random noise and can produce stunning results.

Back in 2015, German researchers from the University of Tubingen discovered that the earlier nodes of neural networks trained on images looked mainly at basic picture elements such as line types, shapes, directions, tones, and colour. These elements collectively made a good representation of "style". This means that by training a smaller neural network on a specific artist's style, we can use an existing image as the starting point to generate new artwork in the style of the artist works use.

Google's Deep Dream Generator was an early adopter of this approach and offered a Deep Style tab to perform this. Many other systems now exist that promise to interpret images, like your selfie, in the style of famous artists, for example.

In this example, I’m using a simple black and white pen sketch of my own, to control the line work but samples the original image colour. Its about what I expected and not exciting, I did however like the unexpected suggestion of a face showing up in the flower on the left.

Created with Google Deep Dream Generator

Style Transfer is available at Night Café, and in these example I used it for two different style reference images. The first, I'm starting with a different flower photo and again using my tree sketch, only this I'm using the option to make a short movie of the iteration involved in "finding" the resulting image.

In my last example I'm using a lesser-known Vincent Van Gogh's still life (why do so many start with Vincent’s work in  their exploration of AIart?)  

This time the results are stronger and closer to something that might be considered good more spontaneous art. But still not really impressive. 

In my next post I will look at trying to get a better colour likeness in the result.

Delving Deeper into the Worlds of AIart

 As the world around us changes rapidly, it's important to keep up with the latest technologies and trends. That's why I decided to delve deeper into the world of AIart, exploring the various offers and services available. Specifically, I've been experimenting with three of the current top AIart tools: Google Deep Dream Generator, Dall.E 2, and Stable Diffusion.

To test out these tools, I've been working with a local group of artists who meet fortnightly on Zoom to paint from a photo. Jeannine Desailly, a co-founder of our little Wednesday Wanderers painting group, kindly supplied a set of photos to paint. I will base my experiment on them.

My first step was to send a suitable prompt “pink lilly still life” to Dall.E 2. While the result was superficially impressive, there were a few things not quite right about the output. The image was overcropped and a little too zoomed in, it lacked the esthetics and composition of a still life subject.

But I'm not giving up on these AIart tools just yet. In my nextpost, I'll be exploring ways to "assist" these tools using these reference images. Stay tuned!

Photowalking on Sunday

Next Photowalk seeing in Detail

The first photowalk has come to an end, with only a few participants showing up. Thanks to those that came But don't worry, we'll make sure to notify you of the next one well in advance. In fact, it's already scheduled on the MGA events calendar!

During this Photowalk, we explored the concept of seeing by chance, inspired by Freeman Patterson's idea of thinking sideways. I’ve provided a short PDF (optimized for phone or iPad) that outlined the tasks for the day. seeing by chance. Firstly, we asked participants to capture three different themes or hashtags. Secondly, we encouraged them to periodically throw a dice and either choose the number of subjects in a photo or the direction they must photograph.

Our objective was to challenge participants to move beyond taking a simple snapshot. With modern cameras and phones, it's easy to obtain well-focused and well-exposed photos. Instead, our random tasks were designed to distract the photowalkers and encourage them to look for different things. By taking numerous photos, participants had the opportunity to review their work later, to see what they captured and saw and how they managed to capture it.

Here are some of my results

#Tranguil, #shadows

#flowers #curve

Six Dots on the Dice

Five Dots

Straight Ahead

Small aside, I forgot to pick up my dice when I took this, but later I was  able to retrace where I took this shot from this photo and there was my dice on the ground!

 

#sculpture #sky

Start planning for the next photowalk, hope to see you there 

There is always time to find your Zoe & your Magic Marker

 

The Artist who Couldn't Draw: an animated film by Danny Gregor

This is a warm little story, I've posted it before we all should listen to and learn. Now its nicely animated, so just watch the video above if you prefer. 

It's never too late, grab the thing you want to do (maybe it's taking better photos, painting, writing, cooking, or climbing mountains) the magic marking is to do it, over and over. Practice and learn to love it.

I think Dan Gregory gives out good advice for free, and Sketchbook Skool is always worth a look.

Can AIart be considered an artistic tool?

Despite the overwhelming impression you might get from just reading the latest social media post that there are two sides to the current #AIart cognitive dissonance.  The artist Luddites, who are claiming their creativityis being stolen versus the non-artistic utopians who think that text to image prompts lets them make great art in seconds. However, there is a lot of ground between the two camps.

I have worked with and spoken to other artists who have been very interested in what this new text to image tools, like Dall-E or Stable Diffusion can do. Many have been investigating these techniques as legitimate ways to create expressions of their unique creativity. In other words, is there life in these approaches as a tool. In my opinion it's not a simple yes/no answer but there is promise.

I have actually been interested and evaluating AI for a long time. I was even very early using the hashtag #AIart. In particular I have experimented with the idea of using such machine learning to capture the essence of my style of mark-making and “find” that within photos I have taken. This is one of the areas of what I have investigating under the “hand drawn photo”project. This method definitely has merit. Below is an example I created at nighcafe using their style transfer method. 

I starred with a handrawn image of an eagle's head I drew for Inktober. This was then used to “train” a simple neural net. I also supplied a photo, a self-portrait, which I had used as inspiration for myrevised profile on instagram. The training of the personalised network has to be scheduled on to suitable computer somewhere about on the internet and does take a while so the resulting image is not returned immediately. It is worth waiting for because now I have a black and white rendering. In my style of inkwork is closely follows the tones and shapes of the photo. I printed it on a laser printer and then hand-coloured it with my earth-coloured Ecoline Brush pens. I would consider this a work of my own art, but I have used some aspects of #Aiart in its creation.

The makeshift Incremental Backup

 My revised archive system is bedding done nicely.

As a gentleman of leisure now, I don’t produce as much data as I used to and thus the automated incremental backups (such as having a NAS system constantly) are no longer urgent and the number of files is considerably smaller. The old NAS has stopped working and the priority of setting up such a system has almost disappeared.

However, in setting up the three-tiered archive system (Son-Father-Grandfather) and re-arranging my back-pack portable harddrives, I ended up with one older 1.5GB that was slightly bulkier than the rest. Significantly it didn’t fit the small zip-up disk wallets I was using to store the archive sets.

I thought it would give me the opportunity to store incremental backups, so I put it through a few chkdsk/fix sessions and all was well. I just kept it on my desk (but not connected) and then at least each week or more often if I doing a lot of stuff (eg larger photo session). All that is required is setting up an INC Backup folder and setting up 5 folders below that for each week in the month, which will ensure I have fewer files to look through should I need to recover anything. I just plug it in and copy anything I’ve been recently working onto the drive.

I’m doing this manually but it is not a big task anymore other than remembering what to copy. Maybe I should set up an automatic file synch system. Hopefully, that can wait.

If a picture is worth 1000 words…

What value will a picture have if it's based on 10, or maybe 100 words?

I’m a chronic dyslexic and words don’t flow easily for me. However, I had decided it was about time to update my Instagram profile picture (it still had me behind a COVID Mask). So why not try and find what Stable Diffusion (a leading text-to-image AI system) had to offer.

Prompt: "Profile artwork for instagram, using watercolour markers"  7 words

Plus a starter image, a head and shoulders photo of me. Hmmm, I’m not a girl and my eyes are roughly the same size. So it didn’t really take much notice of my picture perhaps I need to tell it I have grey hair and a beard

Prompt: "Portrait artist, gray hair goatee beard Profile artwork for instagram, using watercolour markers head and shoulders" 16 words

I add a few negatives  with a -0.3 weight to discourage the process going there

"ugly, scary, poorly drawn face, out of frame, cut off" 10 more words, 26 in total

A mighty improvement and without a starter photo this time BUT how are these an artist and blue rinse in my hair? Seriously?

The last image set was choosen with the same starter but used the additional modifiers (artistic portrait preset image). It added a lot of words some for a positive match some negative, for a total of 88 words

Prompt: "Portrait artist, gray hair goatee beard Profile artwork for instagram, using watercolour markers head and shoulders portrait, 8k resolution concept art portrait by Greg Rutkowski, Artgerm, WLOP, Alphonse Mucha dynamic lighting hyperdetailed intricately detailed Splash art trending on Artstation triadic colors Unreal Engine 5 volumetric lighting” 56words

"ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, extra limbs, disfigured, deformed, body out of frame, blurry, bad anatomy, blurred, watermark, grainy, signature, cut off, draft" 32 negative words

Well it’s a lot more realistically rendered, but still poorly cropped (head chopped up and at best a single shoulder, etc… etc and... Why is this an artist? So do I need to craft a longer more descriptive prompt, or just run a lot more prompts?

The images above have been created on Nightcafe using Stable Diffusion interface

It actually took less time to have a play with my ecoline watercolour brush pens and a pitt pen than to assemble the above.

So my new Instagram profile picture is handdrawn, I trust you understand why.

PS I must humbly apologise to @Greg Rutkowski (I hadn't noticed he was mentioned in the final prompt, hmm the problems with presets) and I do understand why he is not happy.

Why I'm not too concerned about the current #AIart

 

I don't consider these overhyped bitmaps to be magic or even that intelligent. When we see the results of these text to image services, we are looking at a fairly straightforward machine learning neural network application, albeit on a massively large training data set scale.

The data sets are so large and were scrapped together so quickly, without curation or even human review that they contain a lot of straight rubbish and often the darkest side of human behaviour. Whilst this is unpleasant it will be difficult to fix without starting again but there is a taint of not quite right. Importantly, a lot of the material used in the training was copyrighted but that was ignored as it was scrapped and fed into the neural nets. This is legally questionable, but the big money potential may spend this small problem away.

I think it is the lack of ethics in not attributing or even asking artists if they wish to contribute, that could be the undoing of the magic. If the majority of those consuming the images on the net don`t care or have already scrolled over the issue in their cosy little social media bubble, nothing will happen. However, if those looking also see the discussion of unhappy artists and appreciate the widespread collection and indiscriminant use of existing art renderings (style and colour, not cut and paste copies). Perhaps they might investigate and get a whiff of what is so on the nose. Let’s hope so.

Reaching for the light...

How many times have you heard an "expert" claim you can't get bokeh with a M43 (micro four thirds) sensor, zoom lens or stopped down to more than f4? 

Sometimes you just need to ignore those rules.

Opt-in & Opt-out Talk about #AIart and Have I Been Trained website

 Nothing brings the finer points of a debate into the spotlight than personal involvement.

The big AIart news of the last few days is that Stable AI will allow artist to opt out of being included in the next dataset being used to train the neural network to be used for Stable Diffusion 3. Well at least you can opt-out for the next couple of weeks, so follow this up now. 

There is a site, that can check whether you have been included in the massive Laion-5B & Laion-400m neural networks used in Stable Diffusion & Google's Imogen, yes they were trained on 5.8 billion images. I haven't established its legitimacy as an ethical stance, but it does seem legit. It doesn't appear to be a sneaky way to get more images (with so much hacking and phising establishing trust is a big issue in the #AIart discussions)

https://haveibeentrained.com/ 

However, when you see you are one of the artist whose images have been used to train the Lioan Neural Network and had no idea they are in there, some things suddenly confront you.

It took me very little time using the image search feature, to find two of my works. They were part of my Retracing Darwin exhibition in early 2010 and very early examples of my personal technique I call photoimpression. I actually don't mind if others study my method and even create examples of their own. I would like them to acknowledge me, which is becoming a hollow wish on today's web. I don't blame redbubble either, I am sure they didn't know and/or had not given permission either,

I really don't want my work, especially my own special techniques, style, mark-making colouring or composition used without my permission. This is the stuff that makes my work original. Firstly because I know I'll never be acknowledged, that others could profit from this work or contribution to this work to what is presented as their original, I also actually find a lot of the so-called art generated by these AI's a bit scary and I don't approve, and finally I find the whole process a bit morally questionable and not ethical.

So I've made up my mind, Now I do want to opt out.

Now I have to find out how? 

Damn! I have to do it image by image. Cest La Vie

GrandFather, Father, Son archive startegy

It's December already and I'm still contemplating the Backtober idea. Something more distracting seems to arrive everytime I stop to consider a NAS (network storage) option, and I really haven't given trueNAS a real work-out yet either. What I have done is dropped back to the very old Grandfather, Father, Son  Archive strategy from the days when anything off the computer was stored on large reels of magnetic tape. In those days Backup was rightly considered as different to Archive. Usually the backup set was in two parts Full (everything import) and Incremental (only those files that have changed since the last full backup).

What has helped is I am recycling the older USB portable hard drives to store different sets of the data I am now archiving (Largely photos, some finished videos and some PDFs & documents). Swapping them around to fit the file organization. Where once I had the oldest files on the smallest drives (they have been there all their lives) and the newest files on the larger drives (I was moving files to larger drives as I filled them). I now have all the oldest (and theoretically less accessed files on the larger newer drive) and most recent files on the smaller (and older drives). I'm also now convinced that drives need to be exercised (Rather than letting them slowly fail in a cupboard un-noticed so I am now reformating the grandfathers as I recycle them).

The three disks on the green sleeve also act as my Backup set, they are kept on my desk under the shelf/alcove lifting up my monitor to eye level. They are normally not connected to any computer but can be. When I retrieve my grandfather copy from remote storage I immediately clean off the disk (a full format as that does checks of the disk for problems and will map out and mark any problem areas).

I then run copies of the master data and folders on my computer(s) to the freshly formatted disk, There is lots of software to do this, I just used windows backup and restore features built-in windows 10/11. This can take a fair while so best done overnight or in the background. The Incremental Backup can be run manually, just copying files and folders you have been working on to the Incremental disk. This requires a bit of discipline but easy to do at the end of the day or as a task is finished. When I load a batch of photos from my camera I also direct copies to the incremental disk. Alternatively, you can use specialized backup software to automate the process.

AI moving fast and probably in the wrong direction

The developments in AI (well large-scale machine learning in natural language and image generation areas) have been astonishing. However, the "viral" usage seems to be sliding towards that all to common race to the bottom. Well, several bottoms, exploiting others to make money for nothing, chasing fame and likes, selling  AIart as NFTs and now creating and selling books (children's stories in fact). All that as others question the ethics and legality of profiting off someone else's work.

Today I became aware of two books that have been raced into production using these AI tools. (I say this confidentially because the tools to create them have only been available in recent months and the better version in the last weeks)

You can read a wordier (somewhat promotional) story about creating the book on Time (its a pay wall site and wants you to subscribe but you can read this story for free, just ignore the all the ads). The book Alice and Sparkle follows a young girl who builds her own artificial intelligence robot that becomes self-aware and capable of making its own decisions. Ammaar Reshi used ChatGPT, Midjourney and other AI tools were combined to create the book. At the time the article was written he had sold about 70 copies through Amazon since Dec. 4

This was in the normal Blurb email mailout. Guess what it's about dystopian bees. Story by the child of artist Mark Terry and art by artifical intelligence tools. It not exactly cheap but you can get your print on demand version already on Blurb

"I think this book is a glimpse at what anyone can do with merely some AI software, basic Photoshop skills, and an idea. If my [idea] can be turned into a book, then I'm sure your far better ideas can, too!"

—Mark Terry of The Truth About Bees

In the other camp are many unhappy artists ask the question why are these people profiting from our work when we are ignored? Afterall all machine learning system get fed a lot of work created by humans, that's what they learn from. 

So is this straight-out plagiarism? Well not exactly because of the way most Text to image AIs work. They are not directedly copying the "pixel" (or actual marks made) they are learning from a translated space (not the graphic one we see) with an emphasis on things like style, colour choice, composition constructs etc. The Ai then builds the objects "as if" they were painted by a given artist following their style, or a particular photographer or illustrator etc. even just following a generic artistic, cinema, computer game "look". In legal terms this may not be considered as copying (as argued by well-paid lawyers).

What is clearer is the lack of ethics. Profiting (and they can be large profits) from someone else's work and in most cases not even acknowledging them is poor form, not moral and unprincipled. I don't think it is too late to fix this. The argument that the dragon has been let out is valid. Artist should be told (or at least able to find out) if their work in involved in a given neural net and that they can ask their work is removed and the net rebuilt (similar to a take down notice). This will require the licening or similar (we already have creative common licnes) and any the big internet groups applying them to any "content" they make visible on the wider internet. We also need to quosh the idea that anything on the net can be copied and reposted without permission.

Yet I do see that there is great potential for many artist being able to use these Ai tools as aids to improving their skills, helping with inspiration and understanding, and even making their own unique tools on a much smaller scale. As Alice's story says there is power in these tools which can be used for good and evil, depending on how they are guided (what they given to learn)

What can Relight AI do?

It was just a snapshot to show our Christmas tree and decorations to the family, and very ordinary. To dark overall, the dark to light to dark depth confused the autoexposure and the depth of field was a big challenge even for the small aperture used ... etc 

Ok it did the job for the family what's app group but not worthy of sharing outside that.

So I started thinking could Lunimar Neo's new-ish RelightAI tool help out? It has sliders for adding brightness near or far and/or warming to cooling again near or far. Whilst the result is definitely not yet spectacular but it is a definite improvement.

I couldn't help noticing a couple of items lower in the creative block was the Sunrays tool and I just couldn't help myself!

Who said post processing couldn't be fun?

Its about time I had something truly photographic

The weather really picked up a notch today. First real summer weather. Not a cloud in the sky.

So that means the sunset was a little ordinary. BUT still great to watch it set.

You really don't have to photograph everything, just being there can and taking time to gather in the ambience is all you need. Way better than a selfie on your phone that you never look at again.

But a cloudless sky sets a promise of stary night. Forgot it is almost full moon and the bight moon really diminished the star gazing. AND yet just taking the time to view it is also rewarding.

Blood on my hands?

 All I wanted to do was leave Facebook, so I decided to create a goodbye profile picture using a text-to-image AI art algorithm. Really puzzled why I ended up with blood on my hands? 

However, I did make the mistake of making my goodbye message public instead of just to my friends warning them not to click on any request from me, this started a flood of uninformed comments not to be so angry (which I wasn’t just frustrated) and each with convenient links to go somewhere else. If you see those please don’t click on them either. 

So my facebook account should be gone (well in 30days)

When "good enough" isn't really.

The general vibes I am getting back from my fairly intermittent and probably very personalized investigation of the massive deluge of commentary on "text to image" AIart is that of two extremes. Those vehemently against it (with the argument that it will be the end of artists, whatever that means) and those who see it as a wonderful new technology (bringing artistic creation to everyone). I don't agree with either. Like all technology there are pluses and minuses.

Created with Google
Deep Dream Generator

Let's begin by discussing a small personal project of my own. Can AIart tools be used to inspire my creativity? Specifically help me create a new cartoon character, I'm just using a simple prompt.

"cartoon of a watercolour artist painting with red hat"

I submitted this to a number of the more popular AIart tools. I definitely didn't expect similar results as I have been following most of the technical issues how each system has been created. The biggest difference is the data selected to train the neural networks, and remember there are usually two neural nets, one to review images (usually scrapped fairly randomly from the broader web, and the second to analyse associated text (usually within the websites or social media posts as an alternate text caption). Finally, it is very important to recognize that these systems are not copying the original images or even parts of those images. In the case of the image-based network it is just using "abstracted" dimensions (descriptors) that allow it to differentiate or group parts of the image. A previous post includes a video that explains these steps clearly. These could be shape, colour, texts, line ... I'll call them patches of style ... or perhaps how marks are shown might be a better description. Well, the results are a lot different.

Created in Nightcafe using a variety of methods but the same prompt.

First all these differences must make everyone realize that the common generic argument that these are copies or "rip offs" of an artist work, are largely uninformed ("fake new" in social media speak). Clearly, they are different because the neural nets were trained on different & very large datasets. In fact even submitting the same prompts can produce differences within a given tool. They are less copies and more just appropriation of some machine recognizable aspects of the style. You can quickly get into very muddy waters if you try to label this plagiarism, but even so I suspect5 legally this is a grey area.

The second myth to debunk is that because some of these systems are open source and free they are not promoted by greedy profit oriented vested interests (aka wannabe venture capitalist silicon valley types). Despite all this, yes all systems, restrict the number of free prompts you can submit and then offer ways to "buy" more "credits". To make it more game like you can "earn" credits sometimes by continuing to stay on the websites or use a specific part of their services. The really big costs in these systems are not really generating the work (although it might require more power/performance than the average PC or tablet today) it is the collating of the data and neural network training with millions of images and thousands on dimensions for characterization. These big projects requiring many people and seriously massive computer resources. Conveniently, a lot of this data is based on research projects, usually at universities and made available to the community relatively free of restrictions. I don't understand what is going on here, I do suspect some vested interests are deeply involved in academia, but I will restrain my suspicion until I know the work is noble and working towards a better future..

My final consideration for now is, are the photographers, illustrators and artists, whose work has been used in the training of these massive neural networks, happy about their work being included. I doubt they have even been consulted at all. Ok could they opt out? so that their style, characteristic mark making, colouring or composition doesn't keep turning up. I suspect not. The cat is already out of bag?

So, is AIart good enough (in terms of artistic quality)? Of course the quality of art is a very subject matter, and more relevant is the common perception that what is popular must be good. AIart has become very impressive in a short time but it doesn't live up to my expectations at the moment. There is always something a bit off (eg cropping, missing body parts, straying from the brief). Sure, you can refine the prompt, use the magic "words/modifiers", paint in or out, evolve and upscale, if you don't mind being kept using up your credits and buying more. Perhaps it is the public who have become more tolerant of lower resolution imagery and spend a lot less time actually looking at it. Instead, they are keen to scroll onto the next image, looking for that flashy colourful and more often moving picture. Well for a few seconds at least. The "good enough" threshold has perhaps become lower.

Autodraw, another AI assisted drawing tool

I am late discovering this experimental tool from google labs, called autodraw.com. It dates back to mid 2017 and uses the same technology to guess an object as google's quickdraw. Only this time it lets you sketch out the object you want and "It pairs machine learning with drawings from talented artists to help everyone create anything visual, fast." 

Its as simple as that. The graphics are good (created by graphic artists), with Creative Commons Licensing, so free for non-commercial use, easy to download and use. Its also a bit of fun  although you may find it quickly reaches its limits of what it might recognize.

The impulse buy, that wasn't really

 Finally biting the bullet, I had undertaken a little review of what backup/archive space would cost at the moment, specifically on devices I owned. Using USB expansion style hard drives started at around AUD$ 32 to AUD$35 per TB (terabyte=1,000,000 megabytes) or 3.2 to 3.5 cents per GB (gigabyte=1000 megabytes), this was still way cheaper than SSD offering or even old fashion magnetic tape.

A driving force is that I will get close to a digital photo collection (including some movies) approaching 5 TB (terabytes) within the near future. However, my free storage capacity across a lot of hard drives was rapidly reducing below 2TB, and that made it difficult to maintain a 3-2-1 style archive/backup (three copies, on two different devices and one off-site). It has already slipped into a 2-1-1 method (2 copies, on one and a bit devices, one often off-site). I’ve got a couple of 2TB drives already in the backup cycle, so I reasoned two more 3 or 4TB drive would allow me to get back on track. Suddenly I started to see the same ads, from the same seller for such a USB hard drive embedded beside YouTube videos & on between photos on Instagram, damn you targeted marketing! 

But I'm sure I mostly ingored these ads. Until in a different electronic store by chance I saw the 4TB drives again but for a higher price.  I casually mentioned I had seen them cheaper and got an instant price match and even a little extra discount, damn again targeted advertising and slick salemanship. I decided quickly I bought two. 

Now to catch up on the important archiving and backing up that was sliping behind. No more excuses.

Prices have come down a little, as capacities have increased but things are relatively similar to my article back in 2018 The Myth that Digital Photography is free.

Creativity in name, not so much in practice

Whilst I have long since stopped using Lightroom and other Adobe products, I do still take a keen interest in what is going on at Adobe and more particularly what they are working on. So I have been attending the virtual aspects of adobe’s big MAX conference for the last few years.

Adobe CEO Shantanu Narayen described how the company is determined to develop "AI for social good” in his keynote address. They are certainly using AI within many of their tools. Creating a wealth of “one-click”, “fix-it”, “make it better” tools within their existing suites of applications. For sure it will take a lot of tedious hack work out of making visual images and video, but I don’t think this is going to re-assure the real creatives worried about the rise of AI art.

One really obvious theme is everything is moving to video (or at least animated images) and the general idea of creativity is all about building a social media space, going viral, capturing the hearts and mind in a few second, possible in a 2D becoming meta 3D style, design assets and brand kit consistency, again with that “single click” aspect dominating. Sounds like a lot of the sameness, everyone clicking on the same few links and expecting originally and creativity may actually result with more and more of the same!

The hyped “new” tool Adobe Express has a lot of the look and feel of Canva, perhaps with nicer integration with other adobe tools (so you need to pay more rent), The photographic enhancements in lightroom and photoshop are mainly not new and exciting, being offered in a number of other pieces of software, just not integrated so again you have to spend more renting it all. The sneaks segment usually my favourite to see what new technology might be coming up was a disappointment. Perhaps with the exception of project blink (for AI-powered video editing on the web)

I did like the short presentation by Relu Ou on "the myths and realities about Artifical Intelligence", Particularly his closing remark “Please let your imagination direct how you use these tools.”

Second Generation selfportrait

 A lot of people on the Internet express the opinion that artist will go extinct because of #AIart. I am not one of them.

AIart Selfportarit of Artist on the Beach
(Style tranfer of a photo using a stable diffusion reference)

The current crop of text to image neural networks can be quite amazing, very good at capturing style, colouring and texture aspects of artistic mark making. Not so good at expressing an artist intent, specifically originality. Unfortunately, most of the current neural networks I’ve been trained up on rather depressing and dystopian art and images. I’m not sure that this is really the direction of art, more to do with the wasteland of trends on social media, where most images are found these days. Further so much depends on how you express yourself in the text prompt and modifiers. Will poets become the new visual artists?  Further it’s become common practice to keep refining the image and prompt. This is a lot like building up glazes or over-painting, creating layers of added texture and form and more eye appeal.

Already I can see that the simple first attempt text string output on social media from any of the AIart systems around at the moment, looks much the same, not quite right, unusually cropped and frequently scarily dystopian. Hopefully those jumping on the “one-click” filter style of social media influencer posts will soon become bored with the method. Leaving those interested in the potential for artistic expression time to experiment and refine methods. Thereby expanding the range of tools and media they have to hand. I for one look forward to this time.

Inktober-ing to relieve the pain of trying to upload videos from a PC into Reels

I do realize that Instagram's reels are just meta/facebook's copy of Tik/Tok BUT why are they so tediously phone oriented. I have a good little project going where I was submitting the prompt "Am I an Artit Now?" to a variety of common AIart tools and felt it would make an interesting video. Which I still think it will.

However after a week of frustration trying to load & edit a decent reel from my PC, I've realise why should I even bother with such un-socialable media. So sorry folks if you are on instragram, you'll have to put up with my Inktober etchings for a while.

View this post on Instagram

A post shared by Norm Hanson (@normhansonart)

Backtober

I recently listened to a one of Brooks Jensen's "Here's a Thought" short daily podcasts called backtober  Its about how he uses October each year to thoroughly review his digital back up and archives. He might be a bit obsessive, replacing any spinning hard drive older than 3 years, never less it has made me realize I need to totally re-evaluate my situation.

I have not found a good solution to replacing my little NAS unit. So automatic daily backups are not being made anymore!

1. Most of my archiving is now on external (spinning disk) hard drives, and unfortunately, I have experienced 4 of these drives crashing in the past decade. There were duplicates of these so I didn't actually lose anything, Duplicates were on another hard drive (ie same format/technology)
2. I do have archives up to 2010 on CD & DVD, They take up lots of space and I haven't touched them in years.
3. I have a fairly complete "extra" archive on a larger old hard disk that I can connect into my "air gapped" linux archive. When recently accessed one of these older disks started reporting errors (and will have to be replaced, but I'm out of old computer drives!)
4. I haven't evaluated the FreeNAS software yet!

It's October so I need to get onto these now, I like the idea of a full review yearly rather than my irregular monthly attentions (which has been overlooked a few times this year anyway).

Unnatural Crops Explained

The developments in Text to Image AIart generated images are amazing. Things are changing, and largely improving in image quality almost weekly. However one thing I had noticed that was staying fairly constant was unnatural looking crops, the weird truncating of the subjects, particularly people. Surely this was not a new artistic trend I had no knowledge of, or perhaps the artist who's work is being used to train these systems had an aversion to conventional composition.

Example of a headless figure based on stable diffusion prompt

Am I an artist now? John Singer Sargent

Turns out there is a simpler explanation (see quote from a NovelAI blog post below). The unnatural crops are a result of the training set being converted to a square format (so the images are the same ratio) and just arbitrarily using the center of the image.

 Aspect Ratio Bucketing

One common issue of existing image generation models is that they are very prone to producing images with unnatural crops. This is due to the fact that these models are trained to produce square images. However, most photos and artworks are not square. However, the model can only work on images of the same size at the same time, and during training, it is common practice to operate on multiple training samples at once to optimize the efficiency of the GPUs used. As a compromise, square images are chosen, and during training, only the center of each image is cropped out and then shown to the image generation model as a training example.

Are we having fun yet?

It hasn't taken long for my AIart works to fall foul of the censors. Ok not the real censors, the perceived NFW (not for work, standards of the USA based morals on the internet) Whilst I can only see blur in one of the images generated, I expect that it was being interpreted as NUDITY. Which isn't surprising for anything Salvador Dali inspired

I also believe the warning is probably coming from Stable Diffusion itself. That might be a very good thing, indicating they are concerned with the ethics (aka complying with social norms) of the types of images they are being asked to generate. I do know that certain words are not allowed in the text submitted. Again, a bit restrictive but overall, a good thing.

Back to what I was playing with, just having fun. Which I learnt from a Catherine Price in a recent Ted Talk, involves Play, Connection & the Flow State. I like self-referential (meta) subject so I have submitted the phrase "Are we having fun yet?" Firstly the phrase just by itself and the results were totally underwhelming. Perhaps AI doesn't know what fun is yet?

Then by 3 well known artists  / cartoonists / photographers each as modifiers. The results are more encouraging. I've picked the most appropriate in each group of four thumbnails.

Moral: Be careful what you ask for!