基于改进稳定扩散模型的蓝印花布纹样生成

Blue calico pattern generation based on improved stable diffusion model

摘要:

针对非物质文化遗产蓝印花布纹样的数字化传承与创新技术缺乏问题,提出一种基于改进的稳定扩散 (stable diffusion,SD) 模型的蓝印花布纹样生成方法,通过文生图和图生图技术实现蓝印花布单纹样和多纹样的主动生成。对于蓝印花布纹样数据集较少,无法训练出大模型的难题,提出将稳定扩散模型和低秩自适应算法(low-rank adaptation,LoRA)微调网络结合,训练出具有蓝印花布特征的LoRA微调网络；对于稳定扩散模型输出具有随机性问题,提出将稳定扩散模型与判别网络相结合,对生成的图片进行判断,筛选出符合蓝印花布特征的纹理图片。实验结果表明,通过关键提示词或图片,可以生成具有纹样语义信息和图片特征的蓝印花布新纹样。

Abstract:

Aiming at the lack of digital inheritance and innovation technology of blue calico patterns of Chinese intangible cultural heritage,this paper proposes a method for generating blue calico patterns based on the improved stable diffusion (SD) model,which realizing the active generation of single and multiple blue calico patterns through text-to-image and image-to-image technology.For the problem that the blue calico pattern datasets is less and it is difficult to train a large model,a low-rank adaptation algorithm (LoRA) fine-tuning network with blue calico characteristics is trained by combining the stable diffusion model and the LoRA fine-tuning network.Aiming at the problem that the output of the stable diffusion model is random,the stable diffusion model is combined with the discriminative network to judge the generated images and filter out the texture images that are consistent with the characteristics of blue calico.The experimental results show that the new blue calico patterns with semantic information and picture characteristics can be generated by the key prompt words or pictures.