Where Do Art History and Computer Science Meet? (drawing lessons with Amanda Wasielewski)

Hey, Vass.

Hey.

We will hear about you this podcast, I hope. What about your inner artist? Are we going to get that? What kind of art are you into? Does it include computer generated art? Where are you at on this stuff?

It does not include AI generated art. There is a Canadian artist that I'm quite fond of called Jack Bishop who does commentary on Group of Seven work. He does these landscapes of suburbia. He says that's our modern day Group of Seven work.

People should check out, but not everyone on the podcast will know Group of Seven. That is a Canadian gem.

That's right. That's a Canadian gem.

And so people should check it. There's a lot of it online. It is really cool stuff.

I mean, I live with a very small artist. So on Sundays, we go to this wonderful drop in art space for kids for an hour, from 10:00 to 11:00, and we engage together on mixed media and clay and goop. And I'm really good at procuring from the dollar store. If you really want to tip or hot tip, Paul, is these things called paint sticks that you can get at the dollar store. It dries right away. This is the future. The future is now. But what about you? And CIGI has gorgeous art all over. So at the workplace, it's quite nice, but what kind of art are you...

What am I into? I mean, I've always been a lousy artist. I look at other stuff, and I've always been fascinated by paintings, particularly portraits. There's something about those faces in history that are looking at you with a brave face or sad or scared. It's a moment in time has been captured often historically. There's an historical moment or an age or something. And it makes you think about are they capturing...

Did they put the face on they wanted on that artist or was it what that emotion was at that time? And a lot of the time it was, of course, just the artist's view, their eye. Absolutely. But one time when I was looking at this stuff, I was thinking in the earlier digital age, like say 20 years ago or something, it's like where does this go? Is it going to be as good replicated digitally as the real thing?

And it's not really about portraits per se, but it just made me think of could you produce something digitally that somebody could say that's as real as the real thing over there because they're indistinguishable somehow.

Or even when you look at a portrait or a candid, you must have family photos that are in a physical album and you glimpse back at yourself and the way we were. And that feels a little bit different than looking at an image on your cell phone or on your computer that you've taken.

Got lots of those. They were a little bit blurry sometimes. I couldn't tell whether that was the parent's hand because they had a beer in the other hand or because the camera weren't as good. But yeah, I've got a lot of those.

Well, today we're going to speak with an artist and a thinker who is going to help us think through this crossover and interplay between digital and capital A Art. We're speaking with Amanda Wasielewski. Amanda is an associate senior lecturer of digital humanities and an associate professor. She's also a docent of art history in the Department of Archives, Libraries, and Museums at Uppsala University in Sweden. And her artwork has been exhibited internationally. Her most recent monograph is Computational Formalism: Art History and Machine Learning from MIT Press.

I love that title, and the cover, it really got me when I saw it of just kind of like, oh yeah, you just see this screen scanning a face right on the cover.

In our conversation we're going to be exploring the artificial separations between arts and science and how that extends to tensions and conflict between art historians and computer scientists who can learn a lot from each other and who often are.

That's great, Vass. So welcome to Policy Prompt, Amanda. Delighted to have you today.

Amanda Wasielewski:

Thanks so much for having me.

Amanda, it's great to meet you today. Your background is very interesting. You've worked and studied in a few different countries. How has that journey influenced where you are now? Can you give us a bit more about yourself?

Amanda Wasielewski:

Yeah. I mean, I guess I've never really fallen very naturally or neatly into a academic field box. I started out my academic career studying chemistry actually as an undergraduate and decided I wanted to deal with bigger questions in the realm of art. So ended up completely switching directions and going to art. But I still maintained a really strong interest in technology and science.

I worked for a while as a web developer. I had a really mixed background and it made some of my interests a little bit eclectic and unusual for the field of art history, which is the field in which I got my PhD. But ultimately, I think it's really good when you're dealing with cross-disciplinary topics to meet in the middle or try to have some understanding of both sides of the equation.

I think ultimately that interdisciplinary background has really benefited me, have a different perspective, but it's not always easy in the academic realm when you don't very neatly fit into a particular field.

Would you say that computational formalism is unusual or that it doesn't quite fit anywhere because, of course, I have to ask you how you break that down?

Amanda Wasielewski:

Yeah. I mean, you mean the book or the term?

The book and the term. I think I meant to anchor in the term.

Amanda Wasielewski:

Yeah. I think that the book is actually quite grounded in art historical theory and the history of formalism or this term formalism, which is very art historical and it's about the science of forms or how we deal with and how we look at forms. But the whole book is actually about computer science.

It's looking at computer science research from this art historical perspective and hopefully I guess trying to meet halfway in the middle on that to introduce maybe some of these ideas from art history into computer science or at least open up that possibility. But likewise, to open up a better understanding of what's happening in computer science for art history as well.

And in a broader sense, looking at what effect this has in the broader computer vision landscape because art is only one small data set of all of the images that are treated or analyzed or used in computer vision research. I was at a computer vision conference about a month ago with all computer vision researchers, and it was quite interesting because I was the lone humanist there to talk to all these computer vision researchers and often is the case.

They deal with all kinds of different domains. They're dealing with whatever might have visual information, be that medical imagery or entomology or stuff from other sciences or art, other kinds of visual culture items.

Speaker 1:

Hi, everyone. So I'm going to talk about video processing applications. Video is a type of data, type of format that we have every day on our smartphones, on our TVs. And it's unstructured type of data, which contains a lot of information.

Amanda Wasielewski:

My talk had a nice debate at the end of it, and one of the researchers there said, "Amanda, I have to tell you that in computer vision research, most of the researchers don't care about the kind of data that they're using. They want to get their work done that they want to get done, and they don't have time to become domain experts. Some people catch the bug and get really interested in the topic that they're researching, but oftentimes they don't."

And I said, "Yeah, I understand that. I totally understand that that's not the point of the research a lot of the time, but I'm interested in it." I think it's interesting to bring this up in this context because as an art historian, this is my domain knowledge, but I'm also interested in the techniques that you're developing around my domain knowledge. So yeah, so I think that there's a lot to learn from each other across the gulf of disciplinary divides.

So I've seen quite a few different groups interacting before. I don't think I've ever seen art historians and computer scientists in the wild interacting directly, but it feels like to me that with the way generative AI has gone in particular, that things have changed. And as you say, computer scientists use whatever's out there, but they don't think about the origin or the categorization and things that you get into in your book.

And I guess one other surprise with generative AI was it was very powerful on language, but very quickly became powerful on imagery. That surprised me quite a bit in that I didn't think there would be that fundamental generation of images. So it does feel like art history and computer science have been pushed together a lot more closely as a result of where generative AI has gone. Now, do you feel that that's starting to happen more? Are there more contacts and more chats happening?

Amanda Wasielewski:

Oh yeah, 100%. It's been really interesting and quite exciting I think in terms of how are we dealing with these technologies. I mean, there's a lot of hype on both. There's a lot of negative hype. There's a lot of positive hype in the media. There's a lot of artists who are very angry about the rise of generative AI for justifiable reasons.

Speaker 2:

All you got to know is in order for that image to be generated, there has to be a data set. And that is where the problem comes in. In these data sets, there exists billions of copyrighted images of artworks, of photographs, of people, all of which was collected from the internet without the consent of the intellectual property owner. My personal artwork that I share on my Instagram page has been used to train AI models. And chances are...

Amanda Wasielewski:

But you look at the major computer vision research conferences and all of them are introducing art elements, which I think is really fascinating. So artists suddenly are participating. And all the major tech companies have art residencies that relate to AI. So artists working with tech companies as well. I mean, you could think about that really cynically, but I also think there's an opportunity there to broaden the perspectives on these technologies beyond their...

I think a lot of the commercial text to image platforms, their intended use is quite limited. And I think there is more potential there. I don't agree with those critics who say that it has to be some bland, just pastiche of all kinds of stock images. I think there is interesting potential there just to experiment with for artists, and lots of artists are.

So to me that's the interesting part is seeing not only artists interested, because artists are always interested in new technology, artists have often been some of the first early adopters of any new commercial technology to integrate it into artistic practice, but to also see the researchers interested and to see how artists take that on board or what they do with it and even experiment themselves as a...

I mean, there is this utopian idea that this democratizes the making of creative work. And I'm not sure I would go that far, but I think there's definitely an interest in experimenting for a wider group of people that potentially is quite interesting.

Just a follow-up on that, when you look at what's out there on social media, I'm increasingly seeing those prompt engineers asking questions related to images and refining images. You're starting to see it everywhere. It's often not even acknowledged that it's AI generated. It's a background. So it's starting to seep in across the board just these images. And then there's these refined prompt engineers, artistic prompt engineers that are emerging, but there's a lot of division as well. Some people are absolutely not wanting to engage in that space, as you say.

Well, some reject any applications for what is produced. And maybe this is the hype or some subset of that debate with AI generated art, Paul, where it's just like what is this really for? And maybe an artist in residency at a large technology company can help us be thinking through that or experimenting. Amanda, to your mind, what do these roles or chairs or... That's another area where we're merging disciplines in a way. What's the potential there that you think people are missing?

Amanda Wasielewski:

Yeah. I mean, I think we have yet to really unravel what it means to train text and image into a model together. It's not the same thing as a keyword search for an image. There's some kind of synthesis between two different forms that are happening in text image models instead of given the clip based models. I think that there's really interesting potential there, and there's been a long history in humanistic study, text and image study.

So the what is the relationship between text and image? Text is not necessarily just meant to describe images. It interacts with image. It represents things, so do images. There's a semiotics angle to all of this as well. I think artists are willing to explore that and explore the weirdness of the latent space of the model where you'll get certain image fragments, image bits, image types that are trained into a certain semantic area. It's both random and it can be directed.

And I think that's a really interesting aspect for a lot of artists too. I think the explainable AI field has really ascended in computer science research and artists maybe can help with that as well. Not in the technical side of it, but in what does it mean to explain what's happening in a model that otherwise might be a black box that we don't really understand why or how we get out what we get out from certain models given the input we put in.

So I think that that is also potentially... And I mean, I think it's always good to have artists on board in the sense that artists are free agents in a way. They don't tend to have responsibilities to any particular point of view or any particular functionality. So the way that artists use tools is often the way hackers traditionally have used tools is like, I'm going to open this up and I'm going to see how it works.

I'm going to see what it can do. And it's not necessarily doing the things that the people who designed it even envisioned that it would do. People would used to hack different electronics to be instruments or things like that, different devices to do different things. And I think in a similar vein, artists can have the potential to use these systems in that way.

That's not to say that there aren't the very real issues of what do we think about the fact that there are proprietary data sets that use vast amounts of copyrighted material often from artists. You could put their names into a prompt and get out work that looks exactly like this. I don't want to diminish the fact that that is also an issue in the generative AI arena, but I don't think that that means we throw the whole thing away, that there's no potential creative output.

We are speaking with artist and academic Amanda Wasielewski about her book Computational Formalism: Art History and Machine Learning. In it, Amanda explores how the use of machine learning to analyze art images has revived formalism in art history, presenting a golden opportunity for art historians and computer scientists to learn from one another.

You can find Computational Formalism: Art History and Machine Learning at your local bookstore. Just chasing that data set question, I read that your first book, Made in Brooklyn: Artists, Hipsters, Makers, Gentrifiers, is in the book's three data set. I was curious what that's been like for you and maybe even where were you when you figured that out? What was that like?

Amanda Wasielewski:

I think actually my husband told me. Because as soon as that was released online, of course, he went to go search for my book. Honestly, I guess I'm a bit laissez-faire about that kind of stuff. I honestly don't care. I think it's interesting to be part of some kind of collection of big data that is supposed to represent our culture, our society in some way.

And again, that's not to diminish the artists who are upset by the use of their work and that kind of thing, but for me, I see models as these weird little microcosms, these weird little encapsulations of a certain segment of our cultural output. And so to be included in that I guess is to be included in a little mini world that can produce new things. So that's cool, I guess.

So one thing that fascinates me about thinking about computers digitizing art and the history of art is the forgery question. And of course, throughout history, things of value have been forged. Certainly art was key in that and sometimes would even fool experts. Presumably, there are many forgeries that still exist that haven't been detected. But often the grandmasters and others, there were usually ways to authenticity to a high degree of confidence by looking at signature traits in style.

Obviously there was a forensic element to that with canvas and frames and paints and things. We're now seeing huge deepfakes of all kinds, including images. Is there something that makes forgeries very difficult to detect on AI and how big a problem will AI be for generating forgeries?

Amanda Wasielewski:

Yeah. I mean, this is a question that really fascinates me because I think it comes down to also questions about what we value in art, originality, authenticity being attributed to a single person's work. Because of course, a lot of the Renaissance era paintings were still done by workshops, so they might have a name attached to them, but there were many assistants who worked on those works.

And of course, before that time, most art was created by teams or different people working in different capacities. For instance, medieval illumination and things like that. So this idea of the singular genius artist creating the singular work is a relatively recent phenomenon and it's a very Western phenomenon or a Western value. I mean, of course we have a very high...

The art market since the '90s has just exploded, older artworks, but particularly contemporary artwork as well, which there was a recent case of Damien Hirst, the contemporary artist, having backdated a lot of his more recent works, and there's a big scandal about whether... Because that inflates the value of these works, of course, to have them backdated.

Speaker 3:

Hirst's acclaimed formaldehyde animal works, most of which are from the 1990s, it looks like some of them were concocted in 2017. Artificially aged, if you will. And this has left many feeling uneasy. Let's break it down.

Amanda Wasielewski:

Rather than they were made a couple years ago, not 10, 15 years ago. But I mean, I think that this question of AI and authenticity, the companies, the startups that exist that work with using AI to detect what they say are not authentic paintings, I mean, the companies themselves have said this is just one tool in the toolbox. You can't say AI will do it. AI knows for sure whether a painting...

And it's always going to have to be two-dimensional artworks, so I should mention that, in paintings in particular, because a lot of these techniques still use neural networks that are trained to recognize brushstrokes or the hand movement in the same way as a handwriting analysis would, so signature analysis and things like that. And of course, not all artworks subscribe to that, but it also falls under the idea which all art authentication does to a certain extent is that artists won't ever change how their style is enacted.

That artists don't change their style. They don't change their manner of making. That they might not change the materials they use. And of course, we know that's not true. Artists change a lot over their careers. And so the idea is to, okay, how much do they change, but what stays fundamentally the same in an artist's work over the course of their life? The other issue is, of course, that we got issues of data when we deal with authentication, especially automated authentication.

Most artists make very few, in computer data set terms, very few artworks. So the amount of material we have with which to base an authentication conclusion is very little. You don't have millions of artworks to train a neural network on. You don't know how accurate actually it can be. And the other question, of course, is that many of the works we think are authentic are actually not.

There's been some estimates that as many as 20 to 40% of museum collections are either misattributed or completely falsely attributed. These are the questions that our whole system of value, our market value of artworks hinge around, and they're very important questions because there's a lot of money at stake and there's also institutional prestige at stake.

There's one AI authentication startup who has gone around the media saying various artworks and various large collections, national collections are not authentic, and they get good publicity out of that. But the museums, of course, have a big stake in maintaining that those prized works from their collection are in fact authentic.

I mean, the art authentication world has a lot of people with a lot of vested interest, personal interests that don't have anything to do with whether they care if the artwork is actually authentic or not. But I think it is interesting that we can get at... For a certain kind of artwork, we can use neural networks to get close to or at least have an approximation if we think maybe this work was done by the same artist. I mean, that's pretty impressive.

It's pretty cool. It is impressive given those limitations. When a startup is trying to shake up the art world by proclaiming that museum collections are partially falsified or misleading, what are they really pointing at? Because an institution wouldn't knowingly show art that they didn't think was what it was. What are we really talking about in the art world by wanting to pursue this type of verification?

Amanda Wasielewski:

I mean, I do think that institutions once a work has become known and become a draw to their collection might have an interest. If someone came out tomorrow and said, "The Mona Lisa is not a Da Vinci painting," would the Louvre keep showing that painting? Yes, probably a lot of... And they would probably say, "Of course, it is." But I mean, there was the famous case, which is still unresolved, of the painting of Christ, which is said to be of Da Vinci, but then said not to be by various experts.

There was a plan actually to hang that next to the Mona Lisa exhibition at the Louvre, and that didn't actually happen. And then the painting disappeared again. I mean, I think there's a lot of politics involved in the authentication business and obviously a lot of money which complicates things. It's not just the pursuit of truth. And this is true of art historians as well.

There are certain art historians who have made a good deal of money and based their personal reputation on the fact that they said this particular work, I know, I'm an expert, I have the expert eye, I know that this work is authentic. That's their professional reputation, and often that's also their extra paychecks on the line. There's lots of politics involved. And I think entering into that space is also...

Again, it's one of these questions of computer scientists going into a fraught domain that they maybe aren't even fully aware of how fraught that domain is. They just think, oh, hey, I can use this maybe, start a business on art authentication.

And then they enter this whole Wild West World of art authentication, which is the material specialists who are also natural scientists like chemical testing of materials and radiography and stuff like that, and the art historians who claim to have their expert eye, the institutions themselves, and then add to that AI, then you've got a whole lot of different stakeholders saying different things with different motivations in terms of what artwork might or might not be authentic.

With art, both the art itself and the institutions, I think they've underestimated how the inertia and things, as you described. One fundamental issue that nags me a little bit, and you touched on it, is that if you're going to digitize art, so if you're going to take the art and try to put it into a large language model, you've got to digitize it. We all know that digital representation is not the same thing as the real thing like we see with maps.

A map is not the same thing as the terrain that you're walking on. There can be differences. But you've got to do that to get it into a digital form. How realistic is the digital representation? A lot of art, if you think of classic paintings or, as you mentioned, 3D art that is obviously not captured, often art has been designed, say paintings, to reflect certain colors depending on the brightness of the external light and all of these subtle details. So all of that is washed away in a digital version. So we're not actually talking about very representative art in many cases, right?

Amanda Wasielewski:

Yeah. I mean, this is, I guess, one of my key points that I keep harping on about to everyone who will listen, and humanists get it usually. Obviously I think this is something that art historians in particular like to talk about is, what does it matter? How does the materiality change? How does the form of representation change depending on what medium you're translating it to?

There was this famous paper that was released by a computer scientist at NVIDIA, which was proposed a technique called StyleGAN, which was replicated on the website, ThisPersonDoesNotExist.com, which went viral, where every time you refresh the page, you would get a new portrait. The original paper that proposed that technique for StyleGAN, they say in the paper, and this is me being a weird humanist reading computer science papers for the pros, how are they describing what they're doing, which is not what concerns them.

They're all skipping to the method section. But they say, we use the data set of human faces. And I keep bringing this up saying, well, no, you didn't use a data set of human faces. You use a data set of digital photographs of human faces, and it matters that they're photographs and it matters that they're digital. And people were just rolling their eyes at me because for them, it doesn't matter. But it's the same with artworks.

I mean, really what you can represent as a 2D digital image is very limited and far more limited than what art historians would want to discuss when it comes to an artwork. You don't have the scale. You don't have the texture. You don't have the materials. You lose some aspect of detail perhaps. What you're doing when you're analyzing digital databases of digitized artworks is you're analyzing digital images, first and foremost, not artworks.

And so that's not to say though, I mean, art historians have always used different kinds of representations as long as they've existed, photographs, slides to analyze artworks and to compare and understand artworks. It's not to say you can't get information from these representations, but I think that to not acknowledge them, as you said Paul, is it ignores the fact that there are substantial changes and there's also stuff that you just can't encapsulate.

And this is the problem with... I mean, the whole art and computer vision world has a huge problem with... Well, a lot of art after 1960 just cannot be analyzed through these methods because the methods rely on an artwork being contained as a two-dimensional image. So the question is, what's the use of these techniques if they're only for, say, a certain couple centuries of Western art, they're only applicable to this very slim idea of what constitutes art?

I've had these conversations with computer vision researchers saying, "Okay, well, what can we do then? What would be a better representation of art? How can we make data sets or how can we construct data or analyze data that would encapsulate all these factors?" I mean, it's interesting and I'm really glad people have that open mind to be like, okay, if this isn't adequate, then what would be? And I think it's a very complex question. I don't think anyone's satisfactorily found an answer, but at least people are thinking about it.

Policy Prompt is produced by the Centre for International Governance Innovation. CIGI is a nonpartisan think tank based in Waterloo, Canada. With an international network of fellows, experts, and contributors, CIGI tackles the governance challenges and opportunities of data and digital technologies, including AI and their impact on the economy, security, democracy, and ultimately, our societies. Learn more at cigionline.org.

Hey, Vass, are you going to be hanging digital representations of art on your walls anytime soon where you have a digital painting on your wall? Is that going to cut it for you?

No. I have a digital photo frame that changes every five minutes, but those are photos I've put. They are 2D photos, but it's in my place. Would I purchase a piece of digital art in the future? No, that would make me feel like I fell for some kind of scam, to be honest. I don't have the comfort.

Amanda Wasielewski:

You're not going to buy an NFT.

Yeah, I'd be like, oh my God, I guess no one can ever come over again. But I mean, Amanda, you spent a decade working as a practicing artist before shifting to academic work. And I don't mean to present those roles as being mutually exclusive or that you totally left it behind because we opened with your chemical roots, chemistry roots and feeling less traditional or non-traditional. And I was curious. I mean, Paul, I didn't ask you back how much digital art you have. I'm sorry about that. That's rude.

Yeah, it's not rude at all. But I wouldn't put up an original painting of some kind as a digital representation. Would I potentially put up some kind of NFT that was created using current technologies? Maybe. But the idea that I would take something that was drawn by hand, turn it into a digital form and put it on the wall, I don't see that happening.

Amanda Wasielewski:

Print out a JPEG of the Mona Lisa and hang it on above your desk.

I don't know, isn't that any art people bought at IKEA and undergrad or something like this?

Amanda Wasielewski:

Museums sell posters, right? They sell posters all the time.

Truth. Truth.

I mean, that is a way to appreciate art or have something. But yeah, now in terms of what I would source with my very meager and modest art budget, but I want to come back to Amanda and her work as an artist. I wondered, of course, those professions, those pursuits are by no means mutually exclusive.

But I do imagine your rooting as an artist has informed your research and perspectives already. I can hear that coming through when you're like, no, this is what the data set means. Can you tell us more about what it's like to have that reinforcing experience but also be engaged in both worlds?

Amanda Wasielewski:

Yeah. I mean, I think there are certain academic fields where it's not only okay, but normal to both be a practitioner and a researcher, like musicology where most people who enter that field of academic research also have some experience making music or playing instruments or singing or doing music as a practice in some way. Whereas while there are plenty of art historians who might have an art practice background, I would say by and large, it's not accepted that people studying art history have a practice background.

The idea of artistic research has ascended over since the time I did a PhD. I'm intrigued by that idea as well, because on one side, it's a continuation of something that existed in the wild, artists doing research. And my own work was very much research-based, but it also was more open-ended. And I guess that's what I'm trying to get at.

Apart from the strictness of academic life and how you're supposed to conduct academic research there, I think that, I guess, I'm an advocate for methods of artistic research that open it up to something a little bit beyond that to infuse a little bit of creative practice in there. And I think I'm still trying to integrate as much as I can into my academic work. And I think also it helps to have a sensitivity to what are artists' motivations, how do artists work?

I think there's a tendency, especially with fine art, to attribute intentionality to everything. The genius myth of artists always know exactly what they're doing. They're always going to execute the plan that they had in their mind from the beginning. There's no accidents. There's no ulterior motives. It's all just perfectly planned out and beautiful and genius. And I know that's not the case.

I know that's not how artists work. I think there's both a positive side and a more cynical side to be a bit more realistic maybe about what is artistic practice and how can we... Yeah, it definitely 100% informs the research that I do. And also thinking about in the computer science realm, there's a focus on method. There's a focus on practice, but that practice is towards very specific goals.

And I think the potential of art within using these methods is, like I said before, to open it up beyond just the narrow way that researchers might think, okay, I'm developing this technique to do XYZ. But instead, artists can think beyond that. How else can we use it? And I think that is one of the advantages of having a artistic background to start thinking more creatively about how we might use these tools and what they mean.

I'm into it.

Amanda Wasielewski:

But yeah, I mean, I make digital art.

That's why I was like, I would buy it. I don't know.

So thinking about digital art that has generated digital first, let's say, throughout history, the tools that artists use evolve with the times. People couldn't do intricate stone sculptures until they had the right tools to do them. And then that was a huge leap forward for artists in many ways. One of the things that fascinates me is that when digital photography became the norm or the default, there's still some photography that's not digital in the first instance.

But the fact that it became digital raises a very fundamental question, and that is that when an image is taken, whether it's a photograph or a video and it's digital, it's composed of pixels, ultimately just a number of pixels. So how can that distinguish between something that is authentically generated by you, me, or Vass versus something that's generated artificially say by AI or by some nefarious force that's trying to fool people?

They ultimately are just composed of pixels. There was no hand involved, and it's just little dots. So how is one authentic and the other not?

Amanda Wasielewski:

I mean, I think there's a lot of good questions in that. I mean, for artists in the 20th century, many of them from Surrealism through data and neo-data like John Cage and people like Robert Rauschenberg up through the present have harnessed these different elements of chance and automaticness or automatic technologies. And I think from Marcel Duchamp where this idea of de-skilling in art has really been the ascendant concept of artistic practice. On one hand, no pun intended, the hand doesn't really matter. That that kind of...

How many fingers is on the hand? No, I get it.

Amanda Wasielewski:

I mean, so the mark of the hand is no longer the mark of value. I see no conflict with what contemporary art is using any automatic means. I mean, the same debate happened around photography up until the early 20th century. It's like how could photography be art if it's an automatic process, that's light hitting a light sensitive surface. And no one questions now that there is artistic quality to taking photographs.

But on the level of how do you tell the difference, I think that's a more interesting question because, of course, there's lots of people working right now on how do we detect AI generated imagery, not necessarily for art, but for social issues related to deepfakes and things like that, or how to detect if copyrighted material has been used in the data set. And that's a really complex question, because to a certain extent, it's going to be really hard to tell in certain instances.

Because like you said, if it's just on a pixel level, certainly you could say there might be artifacts there in terms of how the image is put together. There's already visible artifacts that we can see, but subsequent programmatic layers on top of models have made a lot of those artifacts better. So asymmetries or the hands problem, which the depicting of hands issue, which many people were frustrated by for the early iterations of text-to-image generators like DALL-E. There's different layers here I think to consider.

And it'll be interesting to see if someone comes up with... Because there's lots of talk of watermarking AI generated imagery, so there's some metadata layer as well that you could look to. But yeah, it would be interesting to see what people come up with in that sense because the big fear is that once all of these AI generated images, if they re-enter the model landscape, then you get a recursion where you're training new image models on AI generated images and it becomes what they call Habsburg AI, the famous royal family in Europe that was very inbred.

So once it's inbred, the genetic defects come to the surface. Well, some researchers have argued or have shown that it will lead to some kind of model collapse. And so it just breaks the system in general. So I think it's a big question in terms of practicality, but it's also a question in terms of materiality, because I do think that the way that AI generated images are put together is so weird and so different than how even the Photoshop paradigm.

You can make all kinds of constructed images with Photoshop, but they have their own artifacts and they have their own way of being that's something different. I think it's super interesting question. It'll be interesting to see what they do, what researchers do with that.

The questions around watermarking and labeling are really fascinating to me because on the one hand, yes, I want to be able to know as a potential consumer or as a viewer of these kinds of artifacts the terms of their production. And on the other hand, I worry or wonder if it ends up just being legitimizing for activities that, again, as you said, based on the copyright or some debates just see all of this as garbage, a form of garbage proliferating all around. I said the G word, garbage. I want to ask you pivoting just a little bit.

Amanda Wasielewski:

It's the enshittification.

Enshittification, baby. I want to ask you about something else you said. Now, this is pulled from social media, so my apologies as a researcher here because we're rooting in your academic work as well, but it was very interesting to me. It's a little bit exasperated, but I think you really had something to say. So the mainstream discourse on AI art and images really drives home how little even well-meaning and smart people know about art history. I saw myself in that. I'm like, oh my god, I'm part of this.

We've had over a century of appropriation as art. So this worry about unoriginality or automation aren't the core issue. The issue has always been corporate use of personal data. I feel like I'm in that category decidedly, and no shame there, just being a little bit basic about art history. How can we improve our own appreciation of art history so we can better participate in these bigger debates about images and art and construction?

Amanda Wasielewski:

I mean, it's hard to say how because there remains a disconnect between the... I mean, what is contemporary art versus what the average person on the street is familiar with in terms of... And I don't want to diminish the people who are illustrators or commercial photographers who will maybe feel that their livelihoods are directly threatened and/or compromised or their copyrighted material is used.

I was talking to someone who heads an organization for artists here in Sweden, and she was saying, "Yeah, absolutely what you said, my artists don't care. They don't care about AI. They don't worry they're going to be replaced because they're contemporary artists and this is not a threat to what contemporary art is." That doesn't mean it's not a threat though to people who are making stock photographs. It's 100% a threat to them.

I think I would like to see the conversation shifted a little bit because I think it's really interesting that, for instance, I don't know if this was present in Canada at all, but there was this famous commercial that would air before movies or before DVDs in the mid-zeros. It was like.

Speaker 4:

You wouldn't steal a handbag. You wouldn't steal a car.

Amanda Wasielewski:

Comparing file sharing basically to all of these what most people thought were... It became a meme because it was so ridiculous.

Speaker 4:

You wouldn't steal a baby.

Amanda Wasielewski:

They were comparing file sharing or downloading movies illegally to stealing a car, robbing a woman of her handbag, breaking into someone's house. This commercial was like, you wouldn't steal a car.

We're all under arrest. We're all under arrest.

Amanda Wasielewski:

I think it's interesting, because looking back to that time, of course, the righteous activists part of society was very much like, no, file sharing isn't equivalent to theft. And I think likewise, the use of data isn't equivalent to theft on the street. It's not going into a gallery and taking a bunch of pictures off the wall. That doesn't mean it isn't copyright infringement, but we could have a whole conversation about how do we enforce copyright.

And I just think it's really interesting that in the mid-zeros, there was the whole copyleft movement. There was the Creative Commons. There's all kinds of things that were trying to push away from this copyright restricted culture landscape. And now it's coming from what you might call the progressive side of things that's pushing for more copyright restrictions. And why is that?

Well, because the people gobbling up all of this, it's not teenagers in their bedrooms downloading movies, it's corporations saying, "Okay, we got all this data. We're using it. We don't care. We don't care what you think about it." The stakes have changed. Who's the bad guy? Who's the good guy? But the fact of it that underpins both of these scenarios is still that we don't know how to value digital materials.

We don't know the value of a digital file. We can't with good faith think to ourselves this is worth something. Corporations don't think so. We as consumers don't think so. The question is how do we deal with our cultural material that exists in digital files? Because over however several decades now, it's been a debate about copyright and the ease of reproducing digital material and how we fundamentally just don't...

We don't value it. I mean, it's become an issue with the streaming culture as well where artists do not receive very much compensation for their streams on Spotify or whatever. I just think that rather than think about it in simple terms of this is plagiarism, this is theft, AI systems are stealing artworks, we can think about it in terms of how do we value digital materials actually, how do we value cultural output that exists in digital form.

The NFT bubble also was an interesting answer or interest in that area too, because it was indeed a bubble. Because ultimately, who's valuing their address on the blockchain versus a JPEG of an ape? I mean, for a brief moment, digital artists were all excited saying, "Oh wow! People finally care about digital arts." And then it all collapsed.

And then Paul bought everything. He's got all his NFTs hanging up.

I actually don't own any NFTs actually, by the way. I might someday. Actually this is a good link to the changes that have literally just happened in the last year or so of just a proliferation of the ability to generate text to image generation, including now video. So the genie is out of the bottle I think in terms of that generation. Regardless of where copyright restrictions may or may not go, there's still going to be a lot of material out there that is not subjected to copyright restrictions.

So one of the core things of your work is to talk about categorization of what it is, what these images and representations are. And that really struck me as quite fascinating, the examples where you said, look, there might be a central figure in let's take a painting for an example, but there might be really interesting things going on in the background, or there might be an animal on the side of the painting or a house or castle or something.

And that that is actually more interesting to some people that would be observing that than the central figure. And so how do you go about trying to categorize these things objectively? And I'm probably not describing it very well here, but can you tell us a little bit about that categorization challenge and how you're thinking about it?

Amanda Wasielewski:

Yeah. I mean, I think one of the most basic fundamental areas of computer vision research is the area of object recognition. And it precedes from the assumption that what we are interested in in the visual field looking at things or interested in images is recognizing what is depicted in that image. So the only reason that a photograph exists is to contain something.

Say you're self-driving. And there's lots of applications, of course, self-driving cars, robotics. You want the image or the video to act as the eye of the computer system. And so it needs to recognize obstacles. It needs to recognize a scene. There's this theory in Gestalt psychology that without a figure and a ground, there is no vision. So you need to be able to see the area of interest from whatever is the background in the image or in the world in order to actually see.

But the problem with this is that there's a lot more to vision, human vision, animal vision, than there is to the camera image. And so to compress all of the things that we see based on our spatial experience, we see as a combination of different senses, we see not an image the way a camera takes an image. We don't see in that way. Our brain puts together things, all these sensory experiences in a synthesis.

We're not just an eyeball floating through the world. And so the problem with object recognition is that it boils down all of the complexity of vision into just this idea of, is there a cat, is there a dog, is there a tree, which are fine things if that's what you're setting out to do. But when you're looking at more complex objects, when you're looking the art objects, that's not necessarily what is the most salient technique to use.

So of course, there's also been attempts at categorizing so-called style, but that comes with a whole other host of problems. Because when we come up with an idea of a category, the categorization automation assumes that these categories are unchangeable, that they're static, that they just exist. They're true/false. They're an on and off. And most things in the world that we categorize are not like that.

Most things are not that simple. So for instance, Kate Crawford and Trevor Paglen did this ImageNet Roulette project that went viral where they showed just how the categorization of human beings can be really problematic in terms of when it was crowdsourced, people labeled humans with all kinds of weird things. Because how do you label humans? What's the most relevant way to categorize people?

And in art history, we debate all the time how do we categorize things. It doesn't mean we don't categorize them, but there's nuances to it that I think are lost in the process of automation. I think categorization is really fundamental here.

Speaking of nuance, your next book is out this fall and it's called Critical Digital Art History. Can you give us just a little bit of a sneak peek or tell us a tiny bit about it?

Amanda Wasielewski:

Yeah, so it's actually an anthology. There's a series of different essays in the book by different authors who are all reflecting on how we can think about the use of digital tools in a more critical way. And so what I mean by critical is not necessarily in a negative way, but in a way that isn't just like, wow, here's cool stuff and here's what we can do with it.

But to think through some of the issues that I've mentioned, issues of different power dynamics or issues of identity or issues of corporate stakeholders or anything like that, so to try to open up this field to all the toolbox of the humanities basically, which should have been there all along and try to think more critically, not necessarily negatively, but critically about how we use digital tools in the field of art history and looking at artworks.

So people in the policy space that might be listening to this podcast, what would their takeaway be? People that are working on data governance issues or AI regulations and things, what is your key message to them to be thinking about?

Amanda Wasielewski:

I mean, I've been involved in a lot of the debates about what's to be done in the creative sector with generative AI. And for me, my top line thing is that we need to be able to see data sets. We need to be able to open up those data sets and know what's in there. The proprietary data set is I think one of the biggest issues because companies like OpenAI... I don't know how you legislate that or design policy around it.

It's going to be a very difficult process. But the idea that OpenAI, their representatives can sit in an interview about Sora, the text to video platform, there was an interview, I don't know if you guys saw it, it went minorly viral, where the interviewer just directly says, "Are there copyrighted YouTube clips?"

Speaker 5:

What data was used to train Sora?

Speaker 6:

We used publicly available data and licensed data.

Speaker 5:

So videos on YouTube?

Speaker 6:

I'm actually not sure about that.

Speaker 5:

Okay. Videos from Facebook, Instagram?

Speaker 6:

If they were publicly available to use, there might be the data, but I'm not sure. I'm not confident about it.

Amanda Wasielewski:

We just collect what's readily available online, and they have their standard lines, but they do all this with impunity because they have proprietary data sets. I don't know if this is the reason, but I assume the reason that Stability AI has had so many lawsuits is because they have used an open source data set so people can see what kind of data is available there for better or worse.

And I mean, that's maybe not possible, but it's at least a worthy goal to understand the data, know what data is in there and how it functions, and then people can know if their work is indeed there. I mean, I think on the flip side, this is my more controversial opinion, but I do think that AI generated image can have artistic value and they're not just completely automated creations.

I think there is artists or author input into them. At the moment, I'm not a huge advocate of copyright. But if other artworks are copyrighted, why not AI-generated artworks? It's a question. I'm not a copyright expert, but I'm a dabbler in copyright. But yeah, I mean, I think that for me as a researcher, it's great that no AI-generated image is under copyright because I can reproduce them in my texts without having to worry about permissions.

But on a larger scale, I do think that if copyright is the mark of, okay, this counts legally as an artwork, then why not extend that to AI-produced artworks that artists have created.

Well, in this AI age, maybe we're all copyright dabblers at the end of the day.

Amanda Wasielewski:

I really do believe that... I guess I'm getting around this, like I was talking about earlier, but somehow we went from this idea of internet culture is all about remixing and sharing and making new things and making memes to being really against all of that. And I understand the reasons why people are against it, but I still think underneath that, this idea that nothing is really "original." Every work we make has influences from all over the place.

No one creates anything from scratch. Everything has some kind of source or inspiration or influence or reacts against something. This is like the history of art is all about that. So I don't know. I feel like we're locking down the potential creativity that could come out of this era because we're, I don't know, justifiably scared, but maybe too scared of some of the consequences of these technologies.

Amanda, it was fantastic to meet you, and your work I think is central to where the debates are going around digitalization, data, AI, everything. And it's been super interesting to hear the artistic side of this and the art history experience that you're bringing to the table. So thanks so much for your time joining us today.

Amanda Wasielewski:

Yeah, thank you for the interesting discussion. Thanks so much.

So Vass, what are your takeaways there?

I really appreciated talking to Amanda because I think she's really challenging us to think about what art is, what it can be, what it should be, how it's evolving, and how even digitally produced images have an element of direction from a human. So those are some of the questions that I am still noodling. What about you?

Yeah, I mean, I thought connecting computer science and art when the rubber hits the road was fascinating because people take a lot of these things for granted. As she said, they're just data points and let's just gather them. And then when you start to drill into it, you realize there's so much nuance and sophistication that will actually never be captured in digitization. So therefore, what's that bridge look like to capture more of that, but it will always be limited.

I think she also touched on some of the broader questions that are out there, like open source versus non-open source, which we're going to come back to a lot, I think, on the show. It's a great example of just where the challenges lie. Looking at art history and computer science was great and I'm sure most people don't spend time thinking about these two linking, so it is really valuable.

We can all spend a little bit more time thinking about it. I also appreciated her sharing with us out the gate that there was a bit of a misfit vibe to her work, feeling that it didn't quite fit neatly anywhere. It strikes me that in academe, but in many places, interdisciplinarity is the buzzword.

It's something we at once aspire to, but then have trouble rewarding because of our goal of categorizing everything. And that challenge of categorization is not unlike what we see in these computational models. We need things to fit or form a pattern, and maybe that at the end of the day just ends up being a little bit unnatural.

Yeah, because they're not totally integrated. As you say, interdisciplinarity, you have a joint meeting, but your title as a professor did not become art scientist from computer scientist or something. It's not actually fully integrated, so the tensions remain.

Or what conferences she goes to, right, because she was saying, speaking what she's noticing about computer scientists. Well, is someone extending the invitation to her, or is she parachuting in because of her own self-declared interest and overlap?

She does seem like a bridge builder to me in that she's open-minded and she's trying to understand both sides, if we could call it that, or say the multiple sides, and has an open mind, which is not the case for others who just say, "Hey, I don't care. I just want data," or others that say, "I reject this outright. There's no way I'm working with generative AI people." I think she's an important player in this space.

Policy Prompt is produced by me, Vass Bednar, and Paul Samson. Tim Lewis and Mel Wiersma are our technical producers. Background research is contributed by Reanne Cayenne. Brand design by Abhilasha Dewan, and creative direction from Som Tsoi. The original theme music is by Joshua Snethlage. Sound Mixing by Francois Goudreau. And special thanks to creative consultant Ken Ogasawara. Please subscribe and rate Policy Prompt wherever you listen to podcasts and stay tuned for future episodes.