使用thumbor搭建独立的图片处理服务
2017-08-18
thumbor介绍 #
图片处理服务是互联网产品开发中必不可少的服务,有很多图片处理引擎如ImageMagic, Graphicsmagic, OpenCV等,我们之前的实现是在我们的对象存储前面搭建了Nginx+Lua+GraphicsMagick的组合,用Lua脚本实现了图片缩放和处理等功能。 最近要搭建独立的图片处理微服务,因此需要对一些图片处理系统做一些调研,就是其中之一。
thumbor是一个小型的图片处理服务,具有缩放、裁剪、翻转、滤镜等功能。thumbor使用python语言开发,内部可以支持Graphicsmagic(pgmagick), OpenCV, PIL (Python Image Library)等图片处理引擎。使用thumbor十分简单,只需要按如下方式的URL访问图片:
1http://thumbor-server/unsafe/300x200/smart/s.glbimg.com/et/bb/f/original/2011/03/24/VN0JiwzmOw0b0lg.jpg
- thumbor的服务thumbor-server通过HTTP从s.glbimg.com获取原生图片,并将其缩放到320x200后返回缩放后的图片
thumbor对图片的存储支持有文件系统、redis、MongoDB,具体可以查看这里Image storage。
对thumbor有了一个初步的了解之后,接下来我们来安装thumbor。
thumbor安装 #
确认python版本:
1python -V
2Python 2.7.5
安装python的包管理工具pip:
1yum install epel-release
2yum install python-devel
3yum install python-pip
安装编译工具:
1yum install gcc
如果需要thumbor处理gif格式的图片,还需要安装gifsicle:
1yum install gifsicle
安装thumbor:
1pip install thumbor
thumbor被安装到了/usr/bin目录下。
下面创建运行thumbor的用户:
1groupadd thumbor
2useradd -g thumbor -s /sbin/nologin thumbor
创建thumbor的配置文件:
1mkdir -p /etc/thumbor
2thumbor-config > /etc/thumbor/thumbor.conf
- 定制相关配置可以参考这里Configuration
创建systemd unit文件/etc/systemd/system/thumbor.service
1[Unit]
2Description=thumbor
3After=network.target
4
5[Service]
6ExecStart=/usr/bin/thumbor \
7 --port=8199 \
8 --conf=/etc/thumbor/thumbor.conf
9User=thumbor
10Restart=on-failure
11Type=simple
12LimitNOFILE=65536
13
14[Install]
15WantedBy=multi-user.target
启动thumbor:
1systemctl status thumbor
2systemctl enable thumbor
nginx配置 #
接下来将thumbor和nginx以及我们的S3对象存储整合到一起。
之前我们S3对象存储中的公开资源是用nginx直接暴露的,例如:
1 upstream s3 {
2 server 192.168.61.113:7480 weight=1 max_fails=2 fail_timeout=30s;
3 server 192.168.61.114:7480 weight=1 max_fails=2 fail_timeout=30s;
4 server 192.168.61.115:7480 weight=1 max_fails=2 fail_timeout=30s;
5 }
6
7 server {
8 listen 80;
9 server_name ~^(?<bucket>.+)\.s3\.frognew\.com$;
10
11 location / {
12 proxy_pass http://s3;
13 rewrite /(.*) /$bucket/$1 break;
14 proxy_set_header Host $host;
15 proxy_set_header X-Real-IP $remote_addr;
16 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
17 }
18 }
用户访问对象存储中的图片资源的URL是类似这样的http://<bucketname>.s3.frognew.com/ChMkJlmWXzqIeD.jpg
。
对nginx的配置文件做如下调整:
1 upstream s3 {
2 server 192.168.61.113:7480 weight=1 max_fails=2 fail_timeout=30s;
3 server 192.168.61.114:7480 weight=1 max_fails=2 fail_timeout=30s;
4 server 192.168.61.115:7480 weight=1 max_fails=2 fail_timeout=30s;
5 }
6
7 upstream thumbor {
8 server 192.168.61.116:8199 weight=1 max_fails=2 fail_timeout=30s;
9 server 192.168.61.117:8199 weight=1 max_fails=2 fail_timeout=30s;
10 }
11
12 server {
13 listen 80;
14 server_name ~^(?<bucket>.+)\.s3\.frognew\.com$;
15
16 location / {
17 proxy_pass http://s3;
18 rewrite /(.*) /$bucket/$1 break;
19 proxy_set_header Host $host;
20 proxy_set_header X-Real-IP $remote_addr;
21 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
22 }
23
24 location ~* ^(.+\.(jpg|jpeg|gif|png))_(\d+x\d+)\.(jpg|jpeg|gif|png)$ {
25 proxy_pass http://thumbor;
26 rewrite ~*/(.+\.(jpg|jpeg|gif|png))_(\d+x\d+)\.(jpg|jpeg|gif|png)$ /unsafe/$3/$bucket.s3.frognew.com/$1 break;
27 proxy_set_header Host $host;
28 proxy_set_header X-Real-IP $remote_addr;
29 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
30 }
31 }
用户希望以缩放的形式访问对象存储的图片资源的URL类似如下格式:
http://<bucketname>.s3.frognew.com/ChMkJlmWXzqIeD.jpg_200x200.jpg
http://<bucketname>.s3.frognew.com/ChMkJlmWXzqIeD.jpg_200x0.jpg
http://<bucketname>.s3.frognew.com/ChMkJlmWXzqIeD.jpg_0x200.jpg
thumbor配置 #
当前对thumbor的配置主要如下:
1MAX_WIDTH = 1024
2MAX_HEIGHT = 1024
3
4ALLOW_ANIMATED_GIFS = False
5
6STORAGE = 'thumbor.storages.file_storage'
7RESULT_STORAGE = 'thumbor.result_storages.file_storage'
8
9FILE_STORAGE_ROOT_PATH = '/home/thumbor/storage'
10STORAGE_EXPIRATION_SECONDS = 648000
11
12RESULT_STORAGE_STORES_UNSAFE = True
13RESULT_STORAGE_EXPIRATION_SECONDS = 648000
14RESULT_STORAGE_FILE_STORAGE_ROOT_PATH = '/home/thumbor/result_storage'