thumbor介绍

图片处理服务是互联网产品开发中必不可少的服务,有很多图片处理引擎如ImageMagic, Graphicsmagic, OpenCV等,我们之前的实现是在我们的对象存储前面搭建了Nginx+Lua+GraphicsMagick的组合,用Lua脚本实现了图片缩放和处理等功能。 最近要搭建独立的图片处理微服务,因此需要对一些图片处理系统做一些调研,就是其中之一。

thumbor是一个小型的图片处理服务,具有缩放、裁剪、翻转、滤镜等功能。thumbor使用python语言开发,内部可以支持Graphicsmagic(pgmagick), OpenCV, PIL (Python Image Library)等图片处理引擎。使用thumbor十分简单,只需要按如下方式的URL访问图片:

1
http://thumbor-server/unsafe/300x200/smart/s.glbimg.com/et/bb/f/original/2011/03/24/VN0JiwzmOw0b0lg.jpg
  • thumbor的服务thumbor-server通过HTTP从s.glbimg.com获取原生图片,并将其缩放到320x200后返回缩放后的图片

thumbor对图片的存储支持有文件系统、redis、MongoDB,具体可以查看这里Image storage

对thumbor有了一个初步的了解之后,接下来我们来安装thumbor。

thumbor安装

确认python版本:

1
2
python -V
Python 2.7.5

安装python的包管理工具pip:

1
2
3
yum install epel-release
yum install python-devel
yum install python-pip

安装编译工具:

1
yum install gcc

如果需要thumbor处理gif格式的图片,还需要安装gifsicle:

1
yum install gifsicle

安装thumbor:

1
pip install thumbor

thumbor被安装到了/usr/bin目录下。

下面创建运行thumbor的用户:

1
2
groupadd thumbor
useradd -g thumbor -s /sbin/nologin thumbor

创建thumbor的配置文件:

1
2
mkdir -p /etc/thumbor
thumbor-config > /etc/thumbor/thumbor.conf

创建systemd unit文件/etc/systemd/system/thumbor.service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
[Unit]
Description=thumbor
After=network.target

[Service]
ExecStart=/usr/bin/thumbor \
	    --port=8199 \
	    --conf=/etc/thumbor/thumbor.conf
User=thumbor
Restart=on-failure
Type=simple
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

启动thumbor:

1
2
systemctl status thumbor
systemctl enable thumbor

nginx配置

接下来将thumbor和nginx以及我们的S3对象存储整合到一起。

之前我们S3对象存储中的公开资源是用nginx直接暴露的,例如:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
    upstream s3 {
        server 192.168.61.113:7480 weight=1 max_fails=2 fail_timeout=30s;
        server 192.168.61.114:7480 weight=1 max_fails=2 fail_timeout=30s;
        server 192.168.61.115:7480 weight=1 max_fails=2 fail_timeout=30s;
    }

    server {
        listen 80;
        server_name ~^(?<bucket>.+)\.s3\.frognew\.com$;

        location / {
            proxy_pass   http://s3;
            rewrite /(.*) /$bucket/$1 break;
            proxy_set_header   Host             $host;
            proxy_set_header   X-Real-IP        $remote_addr;
            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        }
    }

用户访问对象存储中的图片资源的URL是类似这样的http://<bucketname>.s3.frognew.com/ChMkJlmWXzqIeD.jpg

对nginx的配置文件做如下调整:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    upstream s3 {
        server 192.168.61.113:7480 weight=1 max_fails=2 fail_timeout=30s;
        server 192.168.61.114:7480 weight=1 max_fails=2 fail_timeout=30s;
        server 192.168.61.115:7480 weight=1 max_fails=2 fail_timeout=30s;
    }
    
    upstream thumbor {
        server 192.168.61.116:8199 weight=1 max_fails=2 fail_timeout=30s;
        server 192.168.61.117:8199 weight=1 max_fails=2 fail_timeout=30s;
    }

    server {
        listen 80;
        server_name ~^(?<bucket>.+)\.s3\.frognew\.com$;

        location / {
            proxy_pass   http://s3;
            rewrite /(.*) /$bucket/$1 break;
            proxy_set_header   Host             $host;
            proxy_set_header   X-Real-IP        $remote_addr;
            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        }
        
        location ~* ^(.+\.(jpg|jpeg|gif|png))_(\d+x\d+)\.(jpg|jpeg|gif|png)$ {
            proxy_pass   http://thumbor;
            rewrite ~*/(.+\.(jpg|jpeg|gif|png))_(\d+x\d+)\.(jpg|jpeg|gif|png)$ /unsafe/$3/$bucket.s3.frognew.com/$1 break;
            proxy_set_header   Host             $host;
            proxy_set_header   X-Real-IP        $remote_addr;
            proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
        }
    }

用户希望以缩放的形式访问对象存储的图片资源的URL类似如下格式:

  • http://<bucketname>.s3.frognew.com/ChMkJlmWXzqIeD.jpg_200x200.jpg
  • http://<bucketname>.s3.frognew.com/ChMkJlmWXzqIeD.jpg_200x0.jpg
  • http://<bucketname>.s3.frognew.com/ChMkJlmWXzqIeD.jpg_0x200.jpg

thumbor配置

当前对thumbor的配置主要如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
MAX_WIDTH = 1024
MAX_HEIGHT = 1024

ALLOW_ANIMATED_GIFS = False

STORAGE = 'thumbor.storages.file_storage'
RESULT_STORAGE = 'thumbor.result_storages.file_storage'

FILE_STORAGE_ROOT_PATH = '/home/thumbor/storage'
STORAGE_EXPIRATION_SECONDS = 648000

RESULT_STORAGE_STORES_UNSAFE = True
RESULT_STORAGE_EXPIRATION_SECONDS = 648000
RESULT_STORAGE_FILE_STORAGE_ROOT_PATH = '/home/thumbor/result_storage'

参考