Try using vips (#190)

* Try using vips

* No need libaom

* Update code and version

* update

* Fix CI

* Add sudo

* Fix CI

* Make CI happy

* we happy few

* Update README, fix CI

* Add --no-install-recommends

* Update code

* Add ca-certificates for proxymode

* Add pics/invalid.png for testing

* Update README

---------

Co-authored-by: Benny <benny.think@gmail.com>
This commit is contained in:
Nova Kwok 2023-05-10 10:52:36 +08:00 committed by GitHub
parent 882dc98056
commit bfa8aae10c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
18 changed files with 139 additions and 256 deletions

View File

@ -12,8 +12,6 @@ jobs:
ci:
name: CI check on every push
runs-on: ubuntu-latest
container:
image: centos:centos7
steps:
- name: Checkout
uses: actions/checkout@v3
@ -25,9 +23,7 @@ jobs:
- name: Setup necessary packages
run: |
yum install epel-release -y
yum groupinstall 'Development Tools' -y
yum install libaom-devel -y
sudo apt install libvips-dev -y
- name: run test cases
run: make test && make
@ -81,17 +77,17 @@ jobs:
id: trivy
with:
run: |
trivy image --no-progress --severity "HIGH,CRITICAL" ghcr.io/${{ github.event.repository.full_name }}
trivy image --no-progress --severity "HIGH,CRITICAL" --ignore-unfixed ghcr.io/${{ github.event.repository.full_name }}
- name: Print CVE
run: |
echo "${{ steps.trivy.outputs.stdout }}"
- name: Comment PR
uses: thollander/actions-comment-pull-request@v1
uses: thollander/actions-comment-pull-request@v2
with:
message: |
```
${{ steps.trivy.outputs.stdout }}
```
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@ -38,7 +38,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL

View File

@ -13,9 +13,6 @@ jobs:
build:
name: Create Release
runs-on: ubuntu-latest
container:
image: centos:centos7
steps:
- name: Checkout code
uses: actions/checkout@v2
@ -27,9 +24,7 @@ jobs:
- name: Setup necessary packages
run: |
yum install epel-release -y
yum groupinstall 'Development Tools' -y
yum install libaom-devel -y
sudo apt install libvips-dev -y
- name: Make WebP Server Go
run: |

View File

@ -2,7 +2,7 @@ FROM golang:1.20 as builder
ARG IMG_PATH=/opt/pics
ARG EXHAUST_PATH=/opt/exhaust
RUN apt update && apt install libaom-dev -y && mkdir /build
RUN apt update && apt install --no-install-recommends libvips-dev -y && mkdir /build
COPY go.mod /build
RUN cd /build && go mod download
@ -14,11 +14,11 @@ RUN cd /build && sed -i "s|.\/pics|${IMG_PATH}|g" config.json \
FROM debian:bullseye-slim
RUN apt update && apt install --no-install-recommends libvips ca-certificates -y && rm -rf /var/lib/apt/lists/* && rm -rf /var/cache/apt/archives/*
COPY --from=builder /build/webp-server /usr/bin/webp-server
COPY --from=builder /build/config.json /etc/config.json
RUN apt update && apt install libaom-dev -y && rm -rf /var/lib/apt/lists/*
WORKDIR /opt
VOLUME /opt/exhaust
CMD ["/usr/bin/webp-server", "--config", "/etc/config.json"]

View File

@ -12,30 +12,40 @@
This is a Server based on Golang, which allows you to serve WebP images on the fly.
It will convert `jpg,jpeg,png` files by default, this can be customized by editing the `config.json`..
* currently supported image format: JPEG, PNG, BMP, GIF(static image for now)
* currently supported image format: JPEG, PNG, BMP, GIF(static image for now)
> e.g When you visit `https://your.website/pics/tsuki.jpg`it will serve as `image/webp` format without changing the URL.
>
> ~~For Safari and Opera users, the original image will be used.~~
> We've now support Safari/Chrome/Firefox on iOS 14/iPadOS 14
> ~~For Safari and Opera users, the original image will be used.~~
> We've now supported Safari/Chrome/Firefox on iOS 14/iPadOS 14
## Simple Usage Steps(with Binary)
> Note: There is a potential memory leak problem with this server and remains unsolved, we recommend using Docker to mitigate this problem.
> Related discussion: https://github.com/webp-sh/webp_server_go/issues/75
### 1. Prepare the environment
If you'd like to run binary directly on your machine, you need to install `libaom`:
#### If you are using version after 0.6.0
`libaom` is for AVIF support, you can install it by `apt install libaom-dev` on Ubuntu, `yum install libaom-devel` on CentOS.
> Install `libvips` on your machine, more info [here](https://github.com/davidbyttow/govips)
>
> * Ubuntu `apt install libvips-dev`
> * macOS `brew install vips pkg-config`
Without this library, you may encounter error like this: `libaom.so.3: cannot open shared object file: No such file or directory`
#### If you are using version before 0.6.0
If you are using Intel Mac, you can install it by `brew install aom`
If you are using Apple Silicon, you need to `brew install aom && export CPATH=/opt/homebrew/opt/aom/include/;LIBRARY_PATH=/opt/homebrew/opt/aom/lib/`, more references can be found at [在M1 Mac下开发WebP Server Go | 土豆不好吃](https://dmesg.app/m1-aom.html).
If you don't like to hassle around with your system, so do us, why not have a try using Docker? >> [Docker | WebP Server Documentation](https://docs.webp.sh/usage/docker/)
> If you'd like to run binary directly on your machine, you need to install `libaom`:
>
> `libaom` is for AVIF support, you can install it by `apt install libaom-dev` on Ubuntu, `yum install libaom-devel` on CentOS.
>
> Without this library, you may encounter error like this: `libaom.so.3: cannot open shared object file: No such file or directory`
>
> If you are using Intel Mac, you can install it by `brew install aom`
>
> If you are using Apple Silicon, you need to `brew install aom && export CPATH=/opt/homebrew/opt/aom/include/;LIBRARY_PATH=/opt/homebrew/opt/aom/lib/`, more references can be found at [在M1 Mac下开发WebP Server Go | 土豆不好吃](https://dmesg.app/m1-aom.html).
>
> If you don't like to hassle around with your system, so do us, why not have a try using Docker? >> [Docker | WebP Server Documentation](https://docs.webp.sh/usage/docker/)
### 2. Download the binary
@ -96,7 +106,7 @@ For supervisor, Docker sections or detailed Nginx configuration, please read our
## Support us
If you find this project useful, please consider supporting
us [becoming a sponsor](https://github.com/sponsors/webp-sh) or using Stripe
us by [becoming a sponsor](https://github.com/sponsors/webp-sh) or Stripe
| USD(Card, Apple Pay and Google Pay) | SEK(Card, Apple Pay and Google Pay) | CNY(Card, Apple Pay, Google Pay and Alipay) |
|--------------------------------------------------|--------------------------------------------------|--------------------------------------------------|

View File

@ -1,14 +1,10 @@
// webp_server_go - config
// 2020-11-27 13:05
// Benny <benny.think@gmail.com>
package main
type Config struct {
Host string `json:"HOST"`
Port string `json:"PORT"`
ImgPath string `json:"IMG_PATH"`
Quality float32 `json:"QUALITY,string"`
Quality int `json:"QUALITY,string"`
AllowedTypes []string `json:"ALLOWED_TYPES"`
ExhaustPath string `json:"EXHAUST_PATH"`
EnableAVIF bool `json:"ENABLE_AVIF"`
@ -22,8 +18,7 @@ var (
prefetch, proxyMode bool
remoteRaw = "remote-raw"
config Config
version = "0.5.1"
releaseURL = "https://github.com/webp-sh/webp_server_go/releases/latest/download/"
version = "0.6.0"
)
const (

View File

@ -1,23 +1,15 @@
package main
import (
"bytes"
"errors"
"fmt"
"image"
"image/jpeg"
"image/png"
"os"
"path"
"path/filepath"
"runtime"
"strings"
"sync"
"github.com/Kagami/go-avif"
"github.com/chai2010/webp"
"github.com/davidbyttow/govips/v2/vips"
log "github.com/sirupsen/logrus"
"golang.org/x/image/bmp"
)
func convertFilter(raw, avifPath, webpPath string, c chan int) {
@ -73,12 +65,11 @@ func convertImage(raw, optimized, itype string) error {
}
}
//we need to create dir first
// we need to create dir first
err = os.MkdirAll(path.Dir(optimized), 0755)
if err != nil {
log.Error(err.Error())
}
//q, _ := strconv.ParseFloat(config.Quality, 32)
switch itype {
case "webp":
@ -89,79 +80,75 @@ func convertImage(raw, optimized, itype string) error {
return err
}
func readRawImage(imgPath string, maxPixel int) (img image.Image, err error) {
data, err := os.ReadFile(imgPath)
if err != nil {
log.Errorln(err)
}
imgExtension := strings.ToLower(path.Ext(imgPath))
if strings.Contains(imgExtension, "jpeg") || strings.Contains(imgExtension, "jpg") {
img, err = jpeg.Decode(bytes.NewReader(data))
} else if strings.Contains(imgExtension, "png") {
img, err = png.Decode(bytes.NewReader(data))
} else if strings.Contains(imgExtension, "bmp") {
img, err = bmp.Decode(bytes.NewReader(data))
}
if err != nil || img == nil {
errinfo := fmt.Sprintf("image file %s is corrupted: %v", imgPath, err)
log.Errorln(errinfo)
return nil, errors.New(errinfo)
}
x, y := img.Bounds().Max.X, img.Bounds().Max.Y
if x > maxPixel || y > maxPixel {
errinfo := fmt.Sprintf("Read image error: %s(%dx%d) is too large", imgPath, x, y)
log.Warnf(errinfo)
return nil, errors.New(errinfo)
}
return img, nil
}
func avifEncoder(p1, p2 string, quality float32) error {
var img image.Image
dst, err := os.Create(p2)
if err != nil {
log.Fatalf("Can't create destination file: %v", err)
}
// AVIF has a maximum resolution of 65536 x 65536 pixels.
img, err = readRawImage(p1, avifMax)
func avifEncoder(p1, p2 string, quality int) error {
// if convert fails, return error; success nil
var buf []byte
img, err := vips.NewImageFromFile(p1)
if err != nil {
return err
}
err = avif.Encode(dst, img, &avif.Options{
Threads: runtime.NumCPU(),
Speed: avif.MaxSpeed,
Quality: int((100 - quality) / 100 * avif.MaxQuality),
SubsampleRatio: nil,
})
// AVIF has a maximum resolution of 65536 x 65536 pixels.
if img.Metadata().Width > avifMax || img.Metadata().Height > avifMax {
return errors.New("WebP: image too large")
}
// If quality >= 100, we use lossless mode
if quality >= 100 {
buf, _, err = img.ExportAvif(&vips.AvifExportParams{
Lossless: true,
StripMetadata: true,
})
} else {
buf, _, err = img.ExportAvif(&vips.AvifExportParams{
Quality: quality,
Lossless: false,
StripMetadata: true,
})
}
if err != nil {
log.Warnf("Can't encode source image: %v to AVIF", err)
}
if err := os.WriteFile(p2, buf, 0600); err != nil {
log.Error(err)
return err
}
convertLog("AVIF", p1, p2, quality)
return nil
}
func webpEncoder(p1, p2 string, quality float32) error {
func webpEncoder(p1, p2 string, quality int) error {
// if convert fails, return error; success nil
var buf bytes.Buffer
var img image.Image
// The maximum pixel dimensions of a WebP image is 16383 x 16383.
img, err := readRawImage(p1, webpMax)
var buf []byte
img, err := vips.NewImageFromFile(p1)
if err != nil {
return err
}
err = webp.Encode(&buf, img, &webp.Options{Lossless: false, Quality: quality})
// The maximum pixel dimensions of a WebP image is 16383 x 16383.
if img.Metadata().Width > webpMax || img.Metadata().Height > webpMax {
return errors.New("WebP: image too large")
}
// If quality >= 100, we use lossless mode
if quality >= 100 {
buf, _, err = img.ExportWebp(&vips.WebpExportParams{
Lossless: true,
StripMetadata: true,
})
} else {
buf, _, err = img.ExportWebp(&vips.WebpExportParams{
Quality: quality,
Lossless: false,
StripMetadata: true,
})
}
if err != nil {
log.Warnf("Can't encode source image: %v to WebP", err)
}
if err := os.WriteFile(p2, buf.Bytes(), 0600); err != nil {
if err := os.WriteFile(p2, buf, 0600); err != nil {
log.Error(err)
return err
}
@ -170,9 +157,9 @@ func webpEncoder(p1, p2 string, quality float32) error {
return nil
}
func convertLog(itype, p1 string, p2 string, quality float32) {
func convertLog(itype, p1 string, p2 string, quality int) {
oldf, _ := os.Stat(p1)
newf, _ := os.Stat(p2)
log.Infof("%s@%.2f%%: %s->%s %d->%d %.2f%% deflated", itype, quality,
log.Infof("%s@%d%%: %s->%s %d->%d %.2f%% deflated", itype, quality,
p1, p2, oldf.Size(), newf.Size(), float32(newf.Size())/float32(oldf.Size())*100)
}

View File

@ -10,6 +10,8 @@ import (
"testing"
)
var dest = "/tmp/test-result"
func walker() []string {
var list []string
_ = filepath.Walk("./pics", func(p string, info os.FileInfo, err error) error {
@ -23,7 +25,6 @@ func walker() []string {
func TestWebPEncoder(t *testing.T) {
// Go through every files
var dest = "/tmp/test-result"
var target = walker()
for _, f := range target {
runEncoder(t, f, dest)
@ -33,28 +34,25 @@ func TestWebPEncoder(t *testing.T) {
func TestAvifEncoder(t *testing.T) {
// Only one file: img_over_16383px.jpg might cause memory issues on CI environment
var dest = "/tmp/test-result"
avifEncoder("./pics/big.jpg", dest, 80)
assert.Nil(t, avifEncoder("./pics/big.jpg", dest, 80))
assertType(t, dest, "image/avif")
}
func TestNonExistImage(t *testing.T) {
var dest = "/tmp/test-result"
webpEncoder("./pics/empty.jpg", dest, 80)
avifEncoder("./pics/empty.jpg", dest, 80)
assert.NotNil(t, webpEncoder("./pics/empty.jpg", dest, 80))
assert.NotNil(t, avifEncoder("./pics/empty.jpg", dest, 80))
}
func TestConvertFail(t *testing.T) {
var dest = "/tmp/test-result"
webpEncoder("./pics/webp_server.jpg", dest, -1)
avifEncoder("./pics/webp_server.jpg", dest, -1)
func TestHighResolutionImage(t *testing.T) {
assert.NotNil(t, webpEncoder("./pics/img_over_16383px.jpg", dest, 80))
assert.Nil(t, avifEncoder("./pics/img_over_16383px.jpg", dest, 80))
}
func runEncoder(t *testing.T, file string, dest string) {
if file == "pics/empty.jpg" {
t.Log("Empty file, that's okay.")
}
webpEncoder(file, dest, 80)
_ = webpEncoder(file, dest, 80)
assertType(t, dest, "image/webp")
}

17
go.mod
View File

@ -3,16 +3,13 @@ module webp_server_go
go 1.20
require (
github.com/Kagami/go-avif v0.1.0
github.com/chai2010/webp v1.1.0
github.com/davidbyttow/govips/v2 v2.13.0
github.com/gofiber/fiber/v2 v2.4.0
github.com/h2non/filetype v1.1.3
github.com/schollz/progressbar/v3 v3.13.1
github.com/sirupsen/logrus v1.9.0
github.com/staktrace/go-update v0.0.0-20210525161054-fc019945f9a2
github.com/stretchr/testify v1.8.2
github.com/valyala/fasthttp v1.47.0
golang.org/x/image v0.7.0
)
require (
@ -25,12 +22,12 @@ require (
github.com/rivo/uniseg v0.4.3 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/tcplisten v1.0.0 // indirect
golang.org/x/sys v0.6.0 // indirect
golang.org/x/term v0.6.0 // indirect
golang.org/x/image v0.7.0 // indirect
golang.org/x/net v0.8.0 // indirect
golang.org/x/sys v0.7.0 // indirect
golang.org/x/term v0.7.0 // indirect
golang.org/x/text v0.9.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
replace (
github.com/chai2010/webp v1.1.0 => github.com/webp-sh/webp v1.2.0
github.com/gofiber/fiber/v2 v2.4.0 => github.com/webp-sh/fiber/v2 v2.4.0
)
replace github.com/gofiber/fiber/v2 v2.4.0 => github.com/webp-sh/fiber/v2 v2.4.0

28
go.sum
View File

@ -1,22 +1,27 @@
github.com/Kagami/go-avif v0.1.0 h1:8GHAGLxCdFfhpd4Zg8j1EqO7rtcQNenxIDerC/uu68w=
github.com/Kagami/go-avif v0.1.0/go.mod h1:OPmPqzNdQq3+sXm0HqaUJQ9W/4k+Elbc3RSfJUemDKA=
github.com/andybalholm/brotli v1.0.0/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.5 h1:8uQZIdzKmjc/iuPu7O2ioW48L81FgatrcpfFmiq/cCs=
github.com/andybalholm/brotli v1.0.5/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davidbyttow/govips/v2 v2.13.0 h1:5MK9ZcXZC5GzUR9Ca8fJwOYqMgll/H096ec0PJP59QM=
github.com/davidbyttow/govips/v2 v2.13.0/go.mod h1:LPTrwWtNa5n4yl9UC52YBOEGdZcY5hDTP4Ms2QWasTw=
github.com/h2non/filetype v1.1.3 h1:FKkx9QbD7HR/zjK1Ia5XiBsq9zdLi5Kf3zGyFTAFkGg=
github.com/h2non/filetype v1.1.3/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213/go.mod h1:vNUNkEQ1e29fT/6vq2aBdFsgNPmy8qMdSay1npru+Sw=
github.com/klauspost/compress v1.10.7/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.16.3 h1:XuJt9zzcnaz6a16/OU53ZjWp/v7/42WcR5t2a0PcNQY=
github.com/klauspost/compress v1.16.3/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/mattn/go-isatty v0.0.17/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-runewidth v0.0.14 h1:+xnbZSEeDbOIg5/mE6JF0w6n9duR1l3/WmbinWVwUuU=
github.com/mattn/go-runewidth v0.0.14/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db h1:62I3jR2EmQ4l5rM/4FEfDWcRD+abF5XlKShorW5LRoQ=
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db/go.mod h1:l0dey0ia/Uv7NcFFVbCLtqEBQbrT4OCwCSKTEv6enCw=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
@ -26,12 +31,11 @@ github.com/schollz/progressbar/v3 v3.13.1 h1:o8rySDYiQ59Mwzy2FELeHY5ZARXZTVJC7iH
github.com/schollz/progressbar/v3 v3.13.1/go.mod h1:xvrbki8kfT1fzWzBT/UZd9L6GA+jdL7HAgq2RFnO6fQ=
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/staktrace/go-update v0.0.0-20210525161054-fc019945f9a2 h1:kyTDvRL8TyTHOx0aK1PKMPVfkI7rCLDC1nrKF7SYOBc=
github.com/staktrace/go-update v0.0.0-20210525161054-fc019945f9a2/go.mod h1:yzHsbjWRHVF1vVGsluwqCc2TvogFSx9C8TeBh34cMb0=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
@ -47,12 +51,11 @@ github.com/valyala/tcplisten v1.0.0 h1:rBHj/Xf+E1tRGZyWIWwJDiRY0zc1Js+CV5DqwacVS
github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
github.com/webp-sh/fiber/v2 v2.4.0 h1:JtkW0HAqHCExodZMZnG7GrLiJuK2YbNYw8eXo55+tr8=
github.com/webp-sh/fiber/v2 v2.4.0/go.mod h1:f8BRRIMjMdRyt2qmJ/0Sea3j3rwwfufPrh9WNBRiVZ0=
github.com/webp-sh/webp v1.2.0 h1:WiAR1M4Cz50Ehv0vP8VaBZwuTqBgLxcGMaBV23zLVDA=
github.com/webp-sh/webp v1.2.0/go.mod h1:DWmBXPtpA/zfTgEgWxAlsER3B7nXFvDtCi1YY+K5N9w=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/image v0.5.0/go.mod h1:FVC7BI/5Ym8R25iw5OLsgshdUBbT1h5jZTpA+mvAdZ4=
golang.org/x/image v0.7.0 h1:gzS29xtG1J5ybQlv0PuyfE3nmc6R4qB73m6LUUmvFuw=
golang.org/x/image v0.7.0/go.mod h1:nd/q4ef1AKKYl/4kft7g+6UyGbdiqWqTP1ZAbRoV7Rg=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
@ -63,6 +66,9 @@ golang.org/x/net v0.0.0-20201016165138-7b1cca2348c0/go.mod h1:sp8m0HH+o8qH0wwXwY
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.8.0 h1:Zrh2ngAOFYneWTAIAPethzeaQLuHwhuBkuV6ZiRnUaQ=
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@ -77,25 +83,29 @@ golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0 h1:MVltZSvRTcU2ljQOhs94SXPftV6DCNnZViHeQps87pQ=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.7.0 h1:3jlCCIQZPdOYu1h8BkNvLz8Kgwtae2cagcG/VamtZRU=
golang.org/x/sys v0.7.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.6.0 h1:clScbb1cHjoCkyRbWwBEUZ5H/tIFu5TAXIqaZD0Gcjw=
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
golang.org/x/term v0.7.0 h1:BEvjmm5fURWqcfbSKTdpkDXYBrUS1c0m8agp14W48vQ=
golang.org/x/term v0.7.0/go.mod h1:P32HKFT3hSsZrRxla30E9HqToFYAQPCMs/zFMBUFqPY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20200902074654-038fdea0a05b h1:QRR6H1YWRnHb4Y/HeNFCTJLFVxaq6wH4YuVdsUOr75U=
gopkg.in/check.v1 v1.0.0-20200902074654-038fdea0a05b/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@ -21,14 +21,14 @@ import (
)
func avifMatcher(buf []byte) bool {
// 0000001C 66747970 6D696631 00000000 6D696631 61766966 6D696166 000000F4
// 0000001c 66747970 61766966 00000000 61766966 6d696631 6d696166
return len(buf) > 1 && bytes.Equal(buf[:28], []byte{
0x0, 0x0, 0x0, 0x1c,
0x66, 0x74, 0x79, 0x70,
0x6d, 0x69, 0x66, 0x31,
0x0, 0x0, 0x0, 0x0,
0x6d, 0x69, 0x66, 0x31,
0x61, 0x76, 0x69, 0x66,
0x0, 0x0, 0x0, 0x0,
0x61, 0x76, 0x69, 0x66,
0x6d, 0x69, 0x66, 0x31,
0x6d, 0x69, 0x61, 0x66,
})
}

BIN
pics/invalid.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 195 KiB

View File

@ -1,13 +1,10 @@
// webp_server_go - prefetch_test.go
// 2020-11-10 09:27
// Benny <benny.think@gmail.com>
package main
import (
"github.com/stretchr/testify/assert"
"os"
"testing"
"github.com/stretchr/testify/assert"
)
func TestPrefetchImages(t *testing.T) {

View File

@ -1,7 +1,3 @@
// webp_server_go - webp-server_test
// 2020-11-09 11:55
// Benny <benny.think@gmail.com>
package main
import (
@ -76,7 +72,7 @@ func TestConvert(t *testing.T) {
"http://127.0.0.1:3333/webp_server.bmp": "image/webp",
"http://127.0.0.1:3333/webp_server.png": "image/webp",
"http://127.0.0.1:3333/empty.jpg": "",
"http://127.0.0.1:3333/png.jpg": "image/png",
"http://127.0.0.1:3333/png.jpg": "image/webp",
"http://127.0.0.1:3333/12314.jpg": "",
"http://127.0.0.1:3333/dir1/inside.jpg": "image/webp",
"http://127.0.0.1:3333/%e5%a4%aa%e7%a5%9e%e5%95%a6.png": "image/webp",

View File

@ -1,63 +0,0 @@
package main
import (
"encoding/json"
"fmt"
log "github.com/sirupsen/logrus"
"github.com/staktrace/go-update"
"io"
"net/http"
"runtime"
)
func autoUpdate() {
defer func() {
if err := recover(); err != nil {
log.Errorf("Download error: %s", err)
}
}()
var api = "https://api.github.com/repos/webp-sh/webp_server_go/releases/latest"
type Result struct {
TagName string `json:"tag_name"`
}
var res Result
log.Debugf("Requesting to %s", api)
resp1, err := http.Get(api)
if err != nil {
log.Errorf("Error requesting to %s", api)
}
defer resp1.Body.Close()
data1, _ := io.ReadAll(resp1.Body)
_ = json.Unmarshal(data1, &res)
var gitVersion = res.TagName
if gitVersion > version {
log.Infof("Time to update! New version %s found", gitVersion)
} else {
log.Debug("No new version found.")
return
}
var filename = fmt.Sprintf("webp-server-%s-%s", runtime.GOOS, runtime.GOARCH)
if runtime.GOOS == "windows" {
filename += ".exe"
}
log.Info("Downloading binary to update...")
resp, _ := http.Get(releaseURL + filename)
if resp.StatusCode != 200 {
log.Debugf("%s-%s not found on release.", runtime.GOOS, runtime.GOARCH)
return
}
err = update.Apply(resp.Body, update.Options{})
if err != nil {
// error handling
log.Errorf("Update error. %v", err)
} else {
log.Info("Update complete. Please restart to apply changes.")
}
_ = resp.Body.Close()
}

View File

@ -1,37 +0,0 @@
// webp_server_go - update_test
// 2020-11-10 09:36
// Benny <benny.think@gmail.com>
package main
import (
"os"
"testing"
"github.com/stretchr/testify/assert"
)
func TestNormalAutoUpdate(t *testing.T) {
version = "0.0.1"
dir := "./update"
autoUpdate()
assert.NotEqual(t, 0, fileCount(dir))
_ = os.RemoveAll(dir)
}
func Test404AutoUpdate(t *testing.T) {
version = "0.0.1"
dir := "./update"
releaseURL = releaseURL + "a"
autoUpdate()
assert.Equal(t, int64(0), fileCount(dir))
_ = os.RemoveAll(dir)
}
func TestNoNeedAutoUpdate(t *testing.T) {
version = "99.99"
dir := "./update"
autoUpdate()
info, _ := os.Stat(dir)
assert.Nil(t, info)
}

View File

@ -8,6 +8,7 @@ import (
"regexp"
"runtime"
"github.com/davidbyttow/govips/v2/vips"
"github.com/gofiber/fiber/v2"
"github.com/gofiber/fiber/v2/middleware/logger"
log "github.com/sirupsen/logrus"
@ -94,10 +95,14 @@ Develop by WebP Server team. https://github.com/webp-sh`, version)
os.Exit(0)
}
// go autoUpdate()
config = loadConfig(configPath)
switchProxyMode()
vips.Startup(&vips.Config{
ConcurrencyLevel: runtime.NumCPU(),
})
defer vips.Shutdown()
if prefetch {
go prefetchImages(config.ImgPath, config.ExhaustPath)
}

View File

@ -1,17 +1,14 @@
// webp_server_go - webp-server_test
// 2020-11-10 09:41
// Benny <benny.think@gmail.com>
package main
import (
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
"net"
"os"
"runtime"
"testing"
"time"
log "github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
)
// due to test limit, we can't test for cli param part.
@ -21,7 +18,7 @@ func TestLoadConfig(t *testing.T) {
assert.Equal(t, "./exhaust", c.ExhaustPath)
assert.Equal(t, "127.0.0.1", c.Host)
assert.Equal(t, "3333", c.Port)
assert.Equal(t, float32(80), c.Quality)
assert.Equal(t, 80, c.Quality)
assert.Equal(t, "./pics", c.ImgPath)
assert.Equal(t, []string{"jpg", "png", "jpeg", "bmp"}, c.AllowedTypes)
}