Stable Diffusion Webui的QRcode融合實驗紀錄,一個img2img的方法

2023/06/25閱讀時間約 27 分鐘
註:以下僅為本人實驗的紀錄,可能無法一體適用,參數與方法過程僅供有興趣的朋友參考,歡迎有實驗精神的你參與交流。
我用iPhone可以順利快速掃描到網址,但安卓系統就要等一下,不知為何反應稍微會緩慢一些,網址是我工作的醫院(花蓮門諾醫院)的公益募款網站,各位可以安心的嘗試連結。
近期AI在影像生成領域的應用,特別是生產力的支援上,已經有愈來愈多的實際案例,特別是Stable Diffusion WebuiControlnet的協作上(它們的安裝方式亦有許多線上教學)網路上亦有許多專業人士撰文分享經由「動作控制」的創作成果(這裡就不加贅述)而其中一個蠻實際(也蠻有趣)的應用,也常被廣泛討論與實驗的例子,就是透過Controlnet所融合的特殊QRcode影像,透過這個「協同合作」的創作,融合出來的QRcode就不會像傳統二維碼那樣的單調,甚至還帶有濃厚的藝術性質,未來在商業應用上或許也會是一項新的嘗試,以下紀錄僅是我試驗的過程,過程的參數我會放在最後,讓有興趣的朋友研究參考。
這個應用情境可能是,如果你有自己的產品想要以QRcode來行銷,而「文生圖」(txt2img)的結果又無法與你的實際想像契合時,那麼用你的實際產品做為基底的圖生圖(img2img)QRcode就是一個不錯的選項。
以我的例子為例,醫院常有事求人的人員應徵需求,剛好過去又有各職類的棚內拍攝素材留下,而我們過去常有的狀況是,該員離職後,基於尊重的考量,以他為主的拍攝素材就必須撤換(或是重拍過),但是現在可以透過Stable Diffusion與Controlnet的合作,將素材重新運算與重組成新的人物,這也不失為一種高效的工作流程選項之一,各位也不妨做為一種新的嘗試學習看看。

實驗過程

  1. 純化原始照片:照片的來源,是我在醫院實際棚拍的護理人員,接著我將這些實際人物用PS去背處裡,接著用Controlnet的Openpose模組重新運算人物,也將背景進行純化,於是我就有五個新的人物來進行後續的QRcode融合實驗。
  2. 調整照片尺寸與人物比例:照片我除了先以PS編修大小之外(我都改成正方形尺寸以配合QRcode的布局),我也會將人物在畫面的佔比重新調整,因為有時候人物佔比太大,會讓QRcode的準確率大幅降低,有時候在運算過程裡QRcode也比較無法正確「服貼」在人物的衣物上,我試驗的結果是,將人物主體調整成佔畫面的三分之一左右,會讓QRcode的布局完整一些,成功率也會提高許多。
  3. 製作QRcode:將需要宣傳的網址製作成QRcode,尺寸為350x350,容錯率設定在30%,我習慣用這個QRcode平台來產生,當然你也可以用你適合的平台。
  4. 置入圖形:進入Stable Diffusion Webui,選擇圖生圖(img2img),將你的QRcode圖形分別放進圖生圖的圖片欄位,以及Controlnet unit 0的裡面,人物圖形則是放進Controlnet unit 1裡面,Prompt跟Negative欄位你還是要填入適合的人物描繪的說明,愈清楚愈好。
  5. 選擇StableDiffusion的模型與取樣:運算模型繁多,選擇你喜歡的,我自己蠻喜歡moonfilm這個模組,我五張圖也都是用這個模型運算出來的,若你有興趣也可以下載使用。至於取樣(Sampling)我都習慣用DPM++ SDE Karras,你可以選擇你習慣的。
  6. 選擇Controlnet的模型:前面提到的Controlnet unit 0(放QRcode圖)與Controlnet unit 1(放人物圖),都各自有自己的模型需要選擇,Controlnet unit0(放QRcode圖)的預處理器可以選擇none,但模型我是用control_v1p_sd15_qrcode這個專用模型,請你記得前往下載使用。而Controlnet unit 1(放人物圖)的預處理器,我習慣用canny,而模組我習慣用control_v11p_sd15_canny
  7. 調整運算參數:說實在的,各項數值真的需要多多嘗試,沒有一定的規範,我會把五張圖的運算參數結果放在後面讓各位參考,記得看過某位前輩說,這段嘗試與運算的過程,一半是藝術,另一半就算是科學了,只能捲起袖子嘗試看看。
這是將棚拍照片經過Stable Diffusion Webui+Controlnet重新運算過的五張照片,動作與原照片一樣,臉孔也是五位嶄新的虛擬人物。
以下則是我用上面五張運算過的圖片再次運算直出的結果,以iPhone來掃描很快可以得到連結,若以安卓系統掃描就會需要稍微等待一下(為何有這樣的差異?這一點我也不了解...),我把每張的運算參數都附在圖說裡,各位可以發揮實驗精神多方嘗試,最後預祝各位Controlnet之路平安順利。
1 charming girl,perfect face ,asian, Taiwan nurse, medical, medium shot, (masterpiece:1.5), best quality, high resolution, 8K , HDR, Rembrandt lighting,cinematic lighting , bloom, sun light, detailed shadows, rim lighting, film photography, (wind:0.8), (beautiful face:1.5), beautiful girl, ultra detailed eyes, (hyperdetailed:1.15),black eyes, blue surgical uniform, blue shirt, covered mouth, crossed arms, holding own arm, operating cap ,looking at viewer, mask, mouth mask, pink background, shirt, short sleeves, simple background, solo, surgical mask, upper body.
Negative prompt: EasyNegative, (earrings:2.00), (worst quality:2.00), (low quality:2.00), (normal quality:2.00), low-res, (Wearing headphones:2.00), (ugly:2.00)

Steps: 60, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 3865218176, Face restoration: CodeFormer, Size: 768x768, Model hash: 88ea5a84a3, Model: moonfilm_filmGrain10, Denoising strength: 0.75,

ControlNet 0: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1.6, starting/ending: (0.1, 0.7), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)",

ControlNet 1: "preprocessor: canny, model: control_v11p_sd15_canny [d14c016b], weight: 2, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: ControlNet is more important, preprocessor params: (512, 100, 200)",

ControlNet 2: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1.6, starting/ending: (0.85, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)", CFG Rescale phi: 0, Version: v1.3.2
1 charming girl,perfect face,asian, Taiwan nurse, medical, medium shot, (masterpiece:1.5), best quality, high resolution, 8K , HDR, Rembrandt lighting , rim lighting, bokeh, depth of field, film photography, (wind:0.8), beautiful face, (clean face:1.3), white skin, beautiful girl, ultra detailed eyes, (hyperdetailed:1.15), beautiful eyes, black eyes, red surgical uniform, red shirt, covered mouth, crossed arms, holding own arm, operating cap , looking at viewer, (mask:1.2), (mouth mask:1.5), shirt, short sleeves, solo, surgical mask, upper body.

Negative prompt: EasyNegative, (earrings:2.00), (worst quality:2.00), (low quality:2.00), (normal quality:2.00), low-res, (Wearing headphones:2.00), (ugly:2.00)

Steps: 60, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 348821825, Face restoration: CodeFormer, Size: 768x768, Model hash: 88ea5a84a3, Model: moonfilm_filmGrain10, Denoising strength: 0.61,

ControlNet 0: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1.6, starting/ending: (0.1, 0.7), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)",

ControlNet 1: "preprocessor: softedge_pidinet, model: control_v11p_sd15_softedge [a8575a2a], weight: 2, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: ControlNet is more important, preprocessor params: (512, 100, 200)",
ControlNet 2: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1.75, starting/ending: (0.85, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)", CFG Rescale phi: 0, Version: v1.3.2
1 charming girl, perfect face, asian, Taiwan nurse, medical, medium shot, (action of holding a stethoscope:1.2), stethoscope, (masterpiece:1.5), best quality, high resolution, 8K , HDR, Rembrandt lighting, sun light, rim lighting, bokeh, depth of field, film photography, (wind:0.8), detailed hair, (beautiful face:1.5), (clean face:1.3) white skin, beautiful girl, ultra detailed eyes, (hyperdetailed:1.15),brown hair, holding, lips, looking at viewer, nose, realistic, shirt, short hair, simple background, smile, solo, tongue, tongue out, upper body, white shirt

Negative prompt: EasyNegative, (earrings:2.00), (worst quality:2.00), (low quality:2.00), (normal quality:2.00), low-res, (Wearing headphones:2.00), (ugly:2.00)

Steps: 60, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 3348201280, Face restoration: CodeFormer, Size: 768x768, Model hash: 88ea5a84a3, Model: moonfilm_filmGrain10, Denoising strength: 0.75,

ControlNet 0: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1.6, starting/ending: (0.1, 0.7), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)",

ControlNet 1: "preprocessor: canny, model: control_v11p_sd15_canny [d14c016b], weight: 2, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: ControlNet is more important, preprocessor params: (512, 100, 200)",

ControlNet 2: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1.6, starting/ending: (0.85, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)", CFG Rescale phi: 0, Version: v1.3.2
1 charming girl, perfect face ,asian, 22 years old, japanese nurse, medical, (red uniform:1.2), medium shot, (action of holding a stethoscope:1.2),(stethoscope:1.2), (masterpiece:1.5), best quality, high resolution, 8K , HDR, Rembrandt lighting , bloom, detailed shadows, rim lighting, bokeh, depth of field, film photography, (wind:0.8), detailed hair, (beautiful face:1.5), (beautiful girl:1.5), ultra detailed eyes, (hyperdetailed:1.15), brown hair, long sleeves, red cotton t-shirt, (cotton shirt:1.5), smile, solo, teeth, upper body.
Negative prompt: EasyNegative, (earrings:2.00), (worst quality:2.00), (low quality:2.00), (normal quality:2.00), low-res, (Wearing headphones:2.00), (ugly:2.00)

Steps: 60, Sampler: DPM++ SDE Karras, CFG scale: 8, Seed: 1301417385, Face restoration: CodeFormer, Size: 768x768, Model hash: 88ea5a84a3, Model: moonfilm_filmGrain10, Denoising strength: 1,

ControlNet 0: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1.6, starting/ending: (0.1, 0.7), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)",

ControlNet 1: "preprocessor: canny, model: control_v11p_sd15_canny [d14c016b], weight: 2, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: ControlNet is more important, preprocessor params: (512, 100, 200)",

ControlNet 2: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1, starting/ending: (0.85, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)", CFG Rescale phi: 0, Version: v1.3.2
1 charming girl, perfect face, ASIA, Taiwan nurse, (white uniform:1.2), medical, medium shot, (action of holding a stethoscope:1.2), (stethoscope:1.2), (masterpiece:1.5), best quality, high resolution, 8K , HDR, Rembrandt lighting rim lighting, film photography, detailed hair,(beautiful face:1.5), (beautiful girl:1.5), (clean face:1.2), ultra detailed eyes, beautiful eyes, (hyperdetailed:1.4), brown hair, lips, long sleeves, looking at viewer, (simple background:1.2) , nose, shirt, short hair, smile, solo, teeth, upper body, pure white shirt

Negative prompt: EasyNegative, (earrings:2.00), (worst quality:2.00), (low quality:2.00), (normal quality:2.00), low-res, (Wearing headphones:2.00), (ugly:2.00)
Steps: 60, Sampler: DPM++ SDE Karras, CFG scale: 8, Seed: 1736630954, Face restoration: CodeFormer, Size: 768x768, Model hash: 88ea5a84a3, Model: moonfilm_filmGrain10, Denoising strength: 1,

ControlNet 0: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1.6, starting/ending: (0.1, 0.7), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)",

ControlNet 1: "preprocessor: canny, model: control_v11p_sd15_canny [d14c016b], weight: 2, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: ControlNet is more important, preprocessor params: (512, 100, 200)",

ControlNet 2: "preprocessor: none, model: control_v1p_sd15_qrcode [9c780d03], weight: 1, starting/ending: (0.85, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (-1, -1, -1)", CFG Rescale phi: 0, Version: v1.3.2

實驗後記(僅供參)

  1. Controlnet unit0放QR CODE時,若權重低於1.5可能會影響正確辨識率,但表現也會更有創意,大概到1.55左右QRcode正確率會提高。
  2. QRcode與人物圖(或產品圖)的大小,最好都是正方形,主體若是人物最好不要太大,約佔畫面的三分之一左右似乎有最佳結果(掃描成功率)。
  3. 關於Controlnet的控制步數, 人物跟QRcode不要同時算, 最好是人先進, 然後QRcode再進, 這樣服貼在人身上的QRcode會比較自然,一起算的話看起來會平面化,人物圖則是可以從零算到結束(0-1),QRcode圖的進出時間可設定在0.1-0.8之間。
  4. 若有算到滿意的圖像,也可把seed複製下來後,再去修改運算參數,或把Controlnet unit2再加上QRcode圖去算,進出時間可以設定在0.8至1之間。
  5. 以上僅為個人實驗筆記,無法確認正確性,若有失誤都歡迎指正交流。
《如果我的文章對你有任何的幫助或是啟發,非常歡迎你持續追蹤這裡的專題,或是分享給你的朋友,也感謝你的耐心閱讀。
祝你順心 by 福妹可啡工作室
    45會員
    26內容數
    純寫字。
    留言0
    查看全部
    發表第一個留言支持創作者!