Kong Rate-limiting源码解析-1

如题所述

举报该问题

第1个回答 2024-09-19

1. Rate-limiting 简介

Rate-limiting是kong 生态的一款开源限流插件，提供根据时间窗口内请求数量限流的一款插件，其限流策略支持Local、 Redis、cluster, 限流粒度支持： consumer, credential, ip, service, header, path几个级别的限流，限流方式可以用于 Global, Router, Service 。

Docker-compose.yml文件如下：

version: "3.7"services: kong-database:image: postgres:9.6container_name: kong-databaseports:- "5432:5432"environment:- POSTGRES_USER=kong- POSTGRES_DB=kong- POSTGRES_PASSWORD=kongpassnetworks:- kong-netvolumes:- ./docker-data/postgresql/postgresql:/var/lib/postgresql/datakong-gateway:# 镜像版本，目前最新image: kong/kong-gateway:2.8.1.0-alpinecontainer_name: kong-gatewayenvironment:# 数据持久化方式，使用postgres数据库 - "KONG_DATABASE=postgres"# 数据库容器名称,Kong连接数据时使用些名称 - "KONG_PG_HOST=kong-database" - "KONG_PG_USER=kong" - "KONG_PG_PASSWORD=kongpass"# 数据库名称 - "KONG_CASSANDRA_CONTACT_POINTS=kong-database"# 日志记录目录 - "KONG_PROXY_ACCESS_LOG=/dev/stdout" - "KONG_ADMIN_ACCESS_LOG=/dev/stdout" - "KONG_PROXY_ERROR_LOG=/dev/stderr" - "KONG_ADMIN_ERROR_LOG=/dev/stderr"# 暴露的端口 - "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" - "KONG_ADMIN_GUI_URL=http://localhost:8002"ports: - 8000:8000 - 8443:8443 - 8001:8001 - 8444:8444 - 8002:8002 - 8445:8445 - 8003:8003 - 8004:8004# 使用docker网络networks: - kong-net# 依赖数据库服务depends_on: - kong-database# kong 管理界面konga:image: pantsel/kongacontainer_name: kongaenvironment: - "TOKEN_SECRET=51liveup.cn" # 初始化使用development，去初始化数据表，正式使用production - "NODE_ENV=development" - "DB_ADAPTER=postgres"- "DB_HOST=kong-database" - "DB_PORT=5432" - "DB_USER=kong" - "DB_PASSWORD=kongpass" - "DB_DATABASE=konga-db"ports: - 8080:1337networks: - kong-netdepends_on: - kong-databasenetworks:kong-net:external: true

此文件执行启动时会包kong数据库初始化异常,需要执行初始化脚本,注意版本匹配

docker run --rm --network=kong-net-e "KONG_DATABASE=postgres"-e "KONG_PG_HOST=kong-database"-e "KONG_PG_PASSWORD=kongpass" kong/kong-gateway:2.8.1.0-alpine kong migrations bootstrap

具体环境部署会在后期文章中详细讲解

下面进入正题，

因本次再使用Rate-limiting 插件中，需要根据具体某一path 实现限流，其path需要支持通配，故使用了konga UI 界面配置了根据path限流，如下

在测试过程中发现并未达到想要的效果，即 path匹配则限流， path 不匹配则不限流，在多次测试中发现，均使用了Ip 限流，分析起源码发现是插件本身逻辑的问题

插件源码

路径： /usr/local/share/lua/5.1/kong/plugins/rate-limiting

local function get_identifier(conf)local identifierif conf.limit_by == "service" thenidentifier = (kong.router.get_service() orEMPTY).idelseif conf.limit_by == "consumer" thenidentifier = (kong.client.get_consumer() orkong.client.get_credential() orEMPTY).idelseif conf.limit_by == "credential" thenidentifier = (kong.client.get_credential() orEMPTY).idelseif conf.limit_by == "header" thenidentifier = kong.request.get_header(conf.header_name)elseif conf.limit_by == "path" thenlocal req_path = kong.request.get_path()if req_path == conf.path thenidentifier = req_pathendreturn identifier or kong.client.get_forwarded_ip()end

逻辑解析： Rate limiting 只执行时需要根据 rout_id, service_id, indentifier, period_date, period生成一个唯一的local_key进行标识（如下图），而从上面的代码发现 indentifier是根你的限流粒度紧密相关，

从上面的代码可以看出在path匹配判断中，如果path 匹配，将使用req_path作为identifier去后续生成local-key, 但是如果path不匹配，将使用ip 作为identifier，故此时自动回退到使用ip 限流

local get_local_key = function(conf, identifier, period, period_date)local service_id, route_id = get_service_and_route_ids(conf)return fmt("ratelimit:%s:%s:%s:%s:%s", route_id, service_id, identifier, period_date, period)end

2. 改造插件：

由于我的部署方式采用的是docker 部署，故需要先将脚本文件挂载出来，再修改，对docker-compose 配置文件 kong-gateway服务下添加挂载volumes，如下

volumes: - ./docker-data/kong/handler.lua:/usr/local/share/lua/5.1/kong/plugins/rate-limiting/handler.lua

修改handler.lua脚本如下：

local function get_identifier(conf)local identifierif conf.limit_by == "service" thenidentifier = (kong.router.get_service() orEMPTY).idelseif conf.limit_by == "consumer" thenidentifier = (kong.client.get_consumer() orkong.client.get_credential() orEMPTY).idelseif conf.limit_by == "credential" thenidentifier = (kong.client.get_credential() orEMPTY).idelseif conf.limit_by == "header" thenidentifier = kong.request.get_header(conf.header_name)elseif conf.limit_by == "path" thenlocal req_path = kong.request.get_path()--此处可以添加通配处理逻辑，if req_path == conf.path thenidentifier = req_pathendendkong.client.get_forwarded_ip())--去掉默认ip 限流配置，但此时如果path 不匹配 identifier 将会是空（nil）--return identifier or kong.client.get_forwarded_ip()return identifierend

修改此方法如下 local function get_usage(conf, identifier, current_timestamp, limits)

local function get_usage(conf, identifier, current_timestamp, limits)local usage = {}local stop--添加identifier 为空时不进行限流计算（源码没有这个为空校验），在后续处理中，stop为空，将不会进行限流if identifier thenfor period, limit in pairs(limits) dolocal current_usage, err = policies[conf.policy].usage(conf, identifier, period, current_timestamp)if err thenreturn nil, nil, errend-- What is the current usage for the configured limit name?local remaining = limit - current_usage-- Recording usageusage[period] = {limit = limit,remaining = remaining,}if remaining <= 0 thenstop = periodendendendreturn usage, stopend

主方法处理逻辑如下：

function RateLimitingHandler:access(conf)local current_timestamp = time() * 1000-- Consumer is identified by ip address or authenticated_credential idlocal identifier = get_identifier(conf)local fault_tolerant = conf.fault_tolerant-- Load current metric for configured periodlocal limits = {second = conf.second,minute = conf.minute,hour = conf.hour,day = conf.day,month = conf.month,year = conf.year,}local usage, stop, err = get_usage(conf, identifier, current_timestamp, limits)if err thenif not fault_tolerant thenreturn error(err)endkong.log.err("failed to get usage: ", tostring(err))end --将限流信息放进header 中包含信息有: RATELIMIT_LIMIT, RATELIMIT_REMAINING,RATELIMIT_RESET等字段，可以在浏览器端查看if usage then-- Adding headerslocal resetlocal headersif not conf.hide_client_headers thenheaders = {}local timestampslocal limitlocal windowlocal remainingfor k, v in pairs(usage) dolocal current_limit = v.limitlocal current_window = EXPIRATION[k]local current_remaining = v.remainingif stop == nil or stop == k thencurrent_remaining = current_remaining - 1endcurrent_remaining = max(0, current_remaining)if not limit or (current_remaining < remaining) or (current_remaining == remaining and current_window > window)thenlimit = current_limitwindow = current_windowremaining = current_remainingif not timestamps thentimestamps = timestamp.get_timestamps(current_timestamp)endreset = max(1, window - floor((current_timestamp - timestamps[k]) / 1000))endheaders[X_RATELIMIT_LIMIT[k]] = current_limitheaders[X_RATELIMIT_REMAINING[k]] = current_remainingendheaders[RATELIMIT_LIMIT] = limitheaders[RATELIMIT_REMAINING] = remainingheaders[RATELIMIT_RESET] = resetend-- If limit is exceeded, terminate the request-- 如果此处stop 为空将不会进行限流，if stop thenheaders = headers or {}headers[RETRY_AFTER] = resetreturn kong.response.error(429, "API rate limit exceeded", headers)endif headers thenkong.response.set_headers(headers)endendlocal ok, err = timer_at(0, increment, conf, limits, identifier, current_timestamp, 1)if not ok thenkong.log.err("failed to create timer: ", err)endend原文：https://juejin.cn/post/7097143922895355912