赞
踩
温馨提示:内容较长,若想直接看方案,请跳转到总结看即可。(关键字搜索 “整体流程方案”)
最近在项目中遇到个问题,业务须收集客户上传的图片格式证明材料,偶然会发现客户上传成功的图片,无法预览。 开发同学第一时间找运维同学协助排查组件问题,经查实问题是我们所使用的zimg组件作为图片服务器,图片文件存储正常,但是当预览时要生成一个临时文件时失败,导致浏览器客户端无法展示图片。后找客户要了原图,通过微信传输后重新传到zimg服务器,发现展示正常。
##-----------download 下载日志--------------------------- 2022/07/29 11:27:11:231772 [DEBUG] Method: 0 2022/07/29 11:27:11:232173 [DEBUG] Got a GET request for </c2b1b40cb1546c7f8e63597b05f941f7> 2022/07/29 11:27:11:232282 [DEBUG] md5 of request is <c2b1b40cb1546c7f8e63597b05f941f7> 2022/07/29 11:27:11:232320 [DEBUG] get_img() start processing zimg request... 2022/07/29 11:27:11:232346 [DEBUG] whole_path: /root/zimg7/zimg/bin/img/778/109/c2b1b40cb1546c7f8e63597b05f941f7 2022/07/29 11:27:11:232371 [DEBUG] Path[/root/zimg7/zimg/bin/img/778/109/c2b1b40cb1546c7f8e63597b05f941f7] is A Dir. 2022/07/29 11:27:11:232396 [DEBUG] key: c2b1b40cb1546c7f8e63597b05f941f7:0:0:1:0:-1:-1:0:75:jpeg 2022/07/29 11:27:11:232550 [DEBUG] Cache Result: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY 2022/07/29 11:27:11:232580 [DEBUG] Start to Find the Image... 2022/07/29 11:27:11:232603 [DEBUG] 0rig File Path: /root/zimg7/zimg/bin/img/778/109/c2b1b40cb1546c7f8e63597b05f941f7/0*0 2022/07/29 11:27:11:232626 [DEBUG] Got the rsp_path: /root/zimg7/zimg/bin/img/778/109/c2b1b40cb1546c7f8e63597b05f941f7/0*0_p1_g0_-1*-1_r0_q75.jpeg 2022/07/29 11:27:11:232685 [DEBUG] Cache Result: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY 2022/07/29 11:27:11:232711 [DEBUG] Not Hit Original Image Cache. Begin to Open it. 2022/07/29 11:27:11:233388 [DEBUG] Open Original Image From Disk Failed! 0 != 1 # 这里是报错的问题 2022/07/29 11:27:11:233423 [DEBUG] Open Original Image From Disk Failed! 2022/07/29 11:27:11:233452 [DEBUG] zimg Requset Get Image[MD5: c2b1b40cb1546c7f8e63597b05f941f7] Failed! 2022/07/29 11:27:11:233475 [ERROR] 172.16.167.1 fail pic:c2b1b40cb1546c7f8e63597b05f941f7 w:0 h:0 p:1 g:0 x:-1 y:-1 r:0 q:75 f:jpeg 2022/07/29 11:27:11:233507 [DEBUG] ============get_request_cb() ERROR!===============
正常图片上传后 0*0为源文件, 下载时会生成 0 * 0_p1_xxx.jpeg 真实文件, 而HEIC 类型的则无法生成 jpeg 文件, 具体见下方对比图:
目前我们使用的zimg版本是:v3.1.0, ImageMagick版本是:6.9.10-68,系统版本:CentOS 7.9 内核版本:3.10.0-1160.el7.x86_64
# Linux版本
[root@localhost ~]# convert -version
Version: ImageMagick 6.9.10-68 Q16 x86_64 2021-10-14 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP(3.1)
Delegates (built-in): bzlib cairo fontconfig freetype gslib jng jp2 jpeg lcms ltdl lzma openexr pangocairo png ps rsvg tiff wmf x xml zlib
# Mac版本7.1.0-16
~ convert -version
Version: ImageMagick 7.1.0-16 Q16-HDRI x86_64 2021-11-21 https://imagemagick.org
Copyright: (C) 1999-2021 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP(5.0)
Delegates (built-in): bzlib fontconfig freetype gslib heic jng jp2 jpeg lcms lqr ltdl lzma openexr png ps tiff webp xml zlib
Compiler: gcc (4.2)
通过更新 ImageMagick 7.1.0-44,依然无法支持HEIC格式下载:
[root@localhost c2b1b40cb1546c7f8e63597b05f941f7]# convert -version
Version: ImageMagick 7.1.0-44 Q16-HDRI x86_64 20294 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI OpenMP(3.1)
Delegates (built-in): bzlib freetype jng jpeg lzma png tiff x xml zlib
Compiler: gcc (4.8)
卸载ImageMagick, 重新编译安装:无论怎么改,–with-heic的值都为no
后从GitHub获取7.1.0-16 版本,编译安装后,通过指定–with-heic=yes ,可以支持heic
2022/08/01 00:04:18:079828 [DEBUG] Method: 2 2022/08/01 00:04:18:079903 [DEBUG] evblen = 1363658 2022/08/01 00:04:18:080774 [DEBUG] rmblen = 1363658 2022/08/01 00:04:18:080816 [DEBUG] boundary Find. boundary = ----WebKitFormBoundaryxVla4BXhmubWrIuU 2022/08/01 00:04:18:080839 [DEBUG] boundaryPattern = ------WebKitFormBoundaryxVla4BXhmubWrIuU, strlen = 40 2022/08/01 00:04:18:080863 [DEBUG] File[wizard.png] 2022/08/01 00:04:18:080883 [DEBUG] fileType[png] 2022/08/01 00:04:18:080909 [DEBUG] header_value 49 form-data; name="userfile"; filename="wizard.png 2022/08/01 00:04:18:080931 [DEBUG] header_value 9 image/png 2022/08/01 00:04:18:083845 [DEBUG] Begin to Caculate MD5... 2022/08/01 00:04:18:086854 [DEBUG] md5: 951b967ec917a397d5e5111a65cb6501 2022/08/01 00:04:18:086956 [DEBUG] save_path: /root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501 2022/08/01 00:04:18:086994 [DEBUG] Path[/root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501] is Not Existed! 2022/08/01 00:04:18:087370 [DEBUG] save_path[/root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501] Create Finish. 2022/08/01 00:04:18:087404 [DEBUG] save_name-->: /root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501/0*0 2022/08/01 00:04:18:087427 [DEBUG] Start to Storage the New Image... 2022/08/01 00:04:18:088403 [DEBUG] Image [/root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501/0*0] Write Successfully! 2022/08/01 00:04:18:088447 [INFO] 172.16.167.1 succ post pic:951b967ec917a397d5e5111a65cb6501 size:1363471 2022/08/01 00:04:18:088489 [DEBUG] ============post_request_cb() DONE!=============== 2022/08/01 00:04:22:219329 [DEBUG] Method: 0 2022/08/01 00:04:22:219403 [DEBUG] Got a GET request for </951b967ec917a397d5e5111a65cb6501> 2022/08/01 00:04:22:219442 [DEBUG] md5 of request is <951b967ec917a397d5e5111a65cb6501> 2022/08/01 00:04:22:219501 [DEBUG] get_img() start processing zimg request... 2022/08/01 00:04:22:219561 [DEBUG] whole_path: /root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501 2022/08/01 00:04:22:219592 [DEBUG] Path[/root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501] is A Dir. 2022/08/01 00:04:22:219618 [DEBUG] key: 951b967ec917a397d5e5111a65cb6501:0:0:1:0:-1:-1:0:75:none 2022/08/01 00:04:22:219808 [DEBUG] Cache Result: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY 2022/08/01 00:04:22:219838 [DEBUG] Start to Find the Image... 2022/08/01 00:04:22:219860 [DEBUG] 0rig File Path: /root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501/0*0 2022/08/01 00:04:22:219889 [DEBUG] Got the rsp_path: /root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501/0*0_p1_g0_-1*-1_r0_q75.none 2022/08/01 00:04:22:219942 [DEBUG] Cache Result: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY 2022/08/01 00:04:22:219967 [DEBUG] Not Hit Original Image Cache. Begin to Open it. 2022/08/01 00:04:22:270994 [DEBUG] image size = 1363471 2022/08/01 00:04:22:271066 [DEBUG] wi_set_quality(im, 75) 2022/08/01 00:04:22:271094 [DEBUG] convert(im, req) 1 2022/08/01 00:04:22:579246 [DEBUG] Image[/root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501/0*0_p1_g0_-1*-1_r0_q75.none] is Not Existed. Begin to Save it. 2022/08/01 00:04:22:579311 [DEBUG] Start to Storage the New Image... 2022/08/01 00:04:22:580189 [DEBUG] Image [/root/zimg-3.1.0/bin/img/596/741/951b967ec917a397d5e5111a65cb6501/0*0_p1_g0_-1*-1_r0_q75.none] Write Successfully! 2022/08/01 00:04:22:580646 [DEBUG] Begin to Caculate MD5... 2022/08/01 00:04:22:583114 [DEBUG] md5: 4ad1b7cc217480b97fd08a71d80369bf 2022/08/01 00:04:22:583210 [DEBUG] If-None-Match: (null) 2022/08/01 00:04:22:585089 [DEBUG] get buffer length: 1363477 2022/08/01 00:04:22:585128 [DEBUG] Got the File! 2022/08/01 00:04:22:585163 [DEBUG] headers: 1 2022/08/01 00:04:22:585257 [INFO] 172.16.167.1 succ pic:951b967ec917a397d5e5111a65cb6501 w:0 h:0 p:1 g:0 x:-1 y:-1 r:0 q:75 f:none size:1363477 2022/08/01 00:04:22:585316 [DEBUG] ============get_request_cb() DONE!===============
ok , 上传下载PNG没问题!
curl -F "blob=@111.jpg;type=image/jpg" "http://172.16.167.132:9998/upload"
<html>
<head>
<title>Upload Result</title>
</head>
<body>
<h1>MD5: 8f77425b750361a53fec7c83b441ab4f</h1>
Image upload successfully! You can get this image via this address:<br/><br/>
<a href="/8f77425b750361a53fec7c83b441ab4f">http://yourhostname:9998/8f77425b750361a53fec7c83b441ab4f</a>?w=width&h=height&g=isgray&x=position_x&y=position_y&r=rotate&q=quality&f=format
</body>
</html>
测试下载:wget http://172.16.167.133:9998/8f77425b750361a53fec7c83b441ab4f
客户端直接崩溃:
zimg: magick/semaphore.c:347: LockSemaphoreInfo: Assertion `semaphore_info != (SemaphoreInfo *) ((void *)0)' failed.
zimg日志
2022/08/01 00:07:29:771755 [DEBUG] Method: 0 2022/08/01 00:07:29:771843 [DEBUG] Got a GET request for </3bb6ef7ba803f003d9609958bc720f36> 2022/08/01 00:07:29:771871 [DEBUG] md5 of request is <3bb6ef7ba803f003d9609958bc720f36> 2022/08/01 00:07:29:771893 [DEBUG] get_img() start processing zimg request... 2022/08/01 00:07:29:771915 [DEBUG] whole_path: /root/zimg-3.1.0/bin/img/238/443/3bb6ef7ba803f003d9609958bc720f36 2022/08/01 00:07:29:771940 [DEBUG] Path[/root/zimg-3.1.0/bin/img/238/443/3bb6ef7ba803f003d9609958bc720f36] is A Dir. 2022/08/01 00:07:29:771963 [DEBUG] key: 3bb6ef7ba803f003d9609958bc720f36:0:0:1:0:-1:-1:0:75:none 2022/08/01 00:07:29:772199 [DEBUG] Cache Result: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY 2022/08/01 00:07:29:772229 [DEBUG] Start to Find the Image... 2022/08/01 00:07:29:772251 [DEBUG] 0rig File Path: /root/zimg-3.1.0/bin/img/238/443/3bb6ef7ba803f003d9609958bc720f36/0*0 2022/08/01 00:07:29:772272 [DEBUG] Got the rsp_path: /root/zimg-3.1.0/bin/img/238/443/3bb6ef7ba803f003d9609958bc720f36/0*0_p1_g0_-1*-1_r0_q75.none 2022/08/01 00:07:29:772324 [DEBUG] Cache Result: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY 2022/08/01 00:07:29:772349 [DEBUG] Not Hit Original Image Cache. Begin to Open it. 2022/08/01 00:07:29:999538 [DEBUG] image size = 3852068 2022/08/01 00:07:29:999628 [DEBUG] wi_set_quality(im, 75) 2022/08/01 00:07:29:999645 [DEBUG] convert(im, req) 1
上传/下载
[root@localhost ~]# curl -F "blob=@img_20220727_154323.heic;type=image/heic" "http://172.16.167.133:9998/upload" <html> <head> <title>Upload Result</title> </head> <body> <h1>MD5: a7a19c0d70dba023356a92ddfb31c3bf</h1> Image upload successfully! You can get this image via this address:<br/><br/> <a href="/a7a19c0d70dba023356a92ddfb31c3bf">http://yourhostname:9998/a7a19c0d70dba023356a92ddfb31c3bf</a>?w=width&h=height&g=isgray&x=position_x&y=position_y&r=rotate&q=quality&f=format </body> </html> [root@localhost ~]# wget http://172.16.167.133:9998/a7a19c0d70dba023356a92ddfb31c3bf --2022-08-01 00:01:45-- http://172.16.167.133:9998/a7a19c0d70dba023356a92ddfb31c3bf Connecting to 172.16.167.133:9998... connected. HTTP request sent, awaiting response... 404 Not Found 2022-08-01 00:01:45 ERROR 404: Not Found.
下载404日志:
2022/08/01 00:01:45:661914 [DEBUG] Method: 0 2022/08/01 00:01:45:661974 [DEBUG] Got a GET request for </a7a19c0d70dba023356a92ddfb31c3bf> 2022/08/01 00:01:45:662000 [DEBUG] md5 of request is <a7a19c0d70dba023356a92ddfb31c3bf> 2022/08/01 00:01:45:662023 [DEBUG] get_img() start processing zimg request... 2022/08/01 00:01:45:662047 [DEBUG] whole_path: /root/zimg-3.1.0/bin/img/670/103/a7a19c0d70dba023356a92ddfb31c3bf 2022/08/01 00:01:45:662072 [DEBUG] Path[/root/zimg-3.1.0/bin/img/670/103/a7a19c0d70dba023356a92ddfb31c3bf] is A Dir. 2022/08/01 00:01:45:662142 [DEBUG] key: a7a19c0d70dba023356a92ddfb31c3bf:0:0:1:0:-1:-1:0:75:none 2022/08/01 00:01:45:662301 [DEBUG] Cache Conn Failed! 2022/08/01 00:01:45:662329 [DEBUG] Start to Find the Image... 2022/08/01 00:01:45:662351 [DEBUG] 0rig File Path: /root/zimg-3.1.0/bin/img/670/103/a7a19c0d70dba023356a92ddfb31c3bf/0*0 2022/08/01 00:01:45:662374 [DEBUG] Got the rsp_path: /root/zimg-3.1.0/bin/img/670/103/a7a19c0d70dba023356a92ddfb31c3bf/0*0_p1_g0_-1*-1_r0_q75.none 2022/08/01 00:01:45:662491 [DEBUG] Cache Result: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY 2022/08/01 00:01:45:662517 [DEBUG] Not Hit Original Image Cache. Begin to Open it. 2022/08/01 00:01:45:664733 [DEBUG] Open Original Image From Disk Failed! 0 != 1 2022/08/01 00:01:45:664770 [DEBUG] Open Original Image From Disk Failed! 2022/08/01 00:01:45:664843 [DEBUG] zimg Requset Get Image[MD5: a7a19c0d70dba023356a92ddfb31c3bf] Failed! 2022/08/01 00:01:45:664873 [ERROR] 172.16.167.133 fail pic:a7a19c0d70dba023356a92ddfb31c3bf w:0 h:0 p:1 g:0 x:-1 y:-1 r:0 q:75 f:none 2022/08/01 00:01:45:664912 [DEBUG] ============get_request_cb() ERROR!===============
这个是通过微信传输后得来的灵感,通过 thumbnailator 来对原图压缩,来测试压缩后的图片是否被zimg兼容。
<dependency>
<groupId>net.coobird</groupId>
<artifactId>thumbnailator</artifactId>
<version>0.4.8</version>
</dependency>
java代码:
//import org.apache.commons.io.FileUtils /** * 图片压缩方式转换 * @param source * @param dest * @throws IOException */ private static void thumbnailsImg(File source, File dest) throws IOException { byte[] sourceByte = FileUtils.readFileToByteArray(source);//读取源文件 ByteArrayInputStream inputStream = new ByteArrayInputStream(sourceByte); //ByteArrayOutputStream outputStream = new ByteArrayOutputStream(sourceByte.length); Thumbnails.of(inputStream) .scale(0.75) .outputQuality(0.5).toFile(dest);//生成压缩后的新文件 //byte[] bytes = outputStream.toByteArray(); //FileUtils.writeByteArrayToFile(dest, bytes); } //TODO 抛出异常: //Exception in thread "main" net.coobird.thumbnailator.tasks.UnsupportedFormatException: No suitable ImageReader found for source data. //at net.coobird.thumbnailator.tasks.io.InputStreamImageSource.read(Unknown Source) //at net.coobird.thumbnailator.tasks.SourceSinkThumbnailTask.read(Unknown Source) //at net.coobird.thumbnailator.Thumbnailator.createThumbnail(Unknown Source)
这个网上说要把文件先放到ByteArrayInputStream
中再操作,经测试无效,方案失败。
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。