多租户更改

This commit is contained in:
richjohnrambo
2026-08-11 15:59:57 +08:00
parent 6c800de67d
commit c2076721fc
453 changed files with 102349 additions and 47 deletions
+2
View File
@@ -0,0 +1,2 @@
Code in this folder is licensed under Apache 2.0
http://www.apache.org/licenses/LICENSE-2.0
+65
View File
@@ -0,0 +1,65 @@
# Tinode Load Testing
Content of this directory is for running rudimentary load tests of Tinode server. You need this only if you want to run your own load tests.
## Tsung
The `tsung.xml` is a configuration for [Tsung](http://tsung.erlang-projects.org/). The `tinode.beam` is an erlang binary required by the test to generate base64-encoded user-password pairs. The `tinode.erl` is the source for `tinode.beam` (`erlc tinode.erl` -> `tinode.beam`).
[Install Tsung](http://tsung.erlang-projects.org/user_manual/installation.html), then run the test
```
tsung -f ./tsung.xml start
```
## Gatling
A similar loadtest scenario is also available in Gatling. The configuration file is `loadtest.scala`.
Run it with (after [installing Gatling](https://gatling.io/docs/current/installation/)):
```
gatling.sh -sf . -rsf . -rd "na" -s tinode.Loadtest
```
Currently, three tests are available:
* `tinode.Loadtest`: after connecting to server, retrieves user's subscriptions, and publishes a few messages to them one by one.
* `tinode.MeLoadtest`: attempts to max out `me` topic connections.
* `tinode.SingleTopicLoadtest`: connects to and publishes messages to the specified topic (typically, a group topic).
The script supports passing params via the `JAVA_OPTS` envvar.
Parameter name | Default value | Description
-------------- | ------------- | -------------
`num_sessions` | 10000 | Total number of sessions to connect to the server
`ramp` | 300 | Time period in seconds over which to ramp up the load (`0` to `num_sessions`).
`publish_count` | 10 | Number of messages that a user will publish to a topic it subscribes to.
`publish_interval` | 100 | Maximum period of time a user will wait between publishing subsequent messages to a topic.
`accounts` | users.csv | `tinode.Loadtest` and `tinode.SingleTopicLoadtest` only: Path to CSV file containing user accounts to use in loadtest (in format `username,password[,token]` (`token` field is optional).
`topic` | | `tinode.SingleTopicLoadtest` only: topic name to send load to.
`username` | | `tinode.MeLoadtest` only: user to subscribe to `me` topic.
`password` | | `tinode.MeLoadtest` only: user password.
Examples:
```shell
JAVA_OPTS="-Daccounts=users.csv -Dnum_sessions=100 -Dramp=10" gatling.sh -sf . -rsf . -rd "na" -s tinode.Loadtest
```
Ramps up load to 100 sessions listed in `users.csv` file over 10 seconds.
```shell
JAVA_OPTS="-Dusername=user1 -Dpassword=user1123 -Dnum_sessions=10000 -Dramp=600" gatling.sh -sf . -rsf . -rd "na" -s tinode.MeLoadtest
```
Connects 10000 sessions to `me` topic for `user1` with password `user1123` over 600 seconds.
```shell
JAVA_OPTS="-Dtopic=grpYOrcDwORhPg -Daccounts=users.csv -Dnum_sessions=10000 -Dramp=1000 -Dpublish_count=2 -Dpublish_interval=300" gatling.sh -sf . -rsf . -rd "na" -s tinode.SingleTopicLoadtest
```
Connects 10000 users (specified in `users.csv` file) to `grpYOrcDwORhPg` topic over 1000 seconds. Each user will publish 2 messages with interval up to 300 seconds.
This will be eventually packaged into a docker container.
### Experiments
We have tested our single-server Tinode synthetic setup with 50000 accounts on a standard `t3.xlarge` AWS box (4 vCPUs, 16GiB, 5Gbps network) with the `mysql` backend.
As the load increases, before starting to drop:
* The server can sustain 50000 concurrently connected sessions.
* An individual group topic was able to sustain 1500 concurrent sessions.
+133
View File
@@ -0,0 +1,133 @@
package tinode
import java.util.Base64
import java.util.concurrent.ConcurrentHashMap
import scala.collection.JavaConverters._
import scala.collection._
import scala.concurrent.duration._
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class Loadtest extends TinodeBase {
// Input file can be set with the "accounts" java option.
// E.g. JAVA_OPTS="-Daccounts=/tmp/z.csv" gatling.sh -sf . -rsf . -rd "na" -s tinode.Loadtest
val feeder = csv(System.getProperty("accounts", "users.csv")).random
val scn = scenario("WebSocket")
.exec(ws("Connect WS").connect("/v0/channels?apikey=AQEAAAABAAD_rAp4DJh05a1HAwFT3A6K"))
.exec(session => session.set("id", "tn-" + session.userId))
.pause(1)
.exec(hello)
.pause(1)
.feed(feeder)
.doIfOrElse({session =>
val uname = session("username").as[String]
var token = session("token").asOption[String]
if (token == None) {
token = tokenCache.get(uname)
}
token == None
}) { loginBasic } { loginToken }
.exitHereIfFailed
.exec(subMe)
.exitHereIfFailed
.exec(getSubs)
.exitHereIfFailed
.doIf({session =>
session.attributes.contains("subs")
}) {
exec { session =>
// Shuffle subscriptions.
val subs = session("subs").as[Vector[String]]
val shuffled = scala.util.Random.shuffle(subs.toList)
session.set("subs", shuffled)
}
.foreach("${subs}", "sub") {
exec(subTopic)
.exitHereIfFailed
.pause(0, 2)
.doIfOrElse({session =>
val topic = session("sub").as[String]
!topic.startsWith("chn")
}) { publish } { pause(5) }
.exec(leaveTopic)
.pause(0, 3)
}
}
.exec(ws("close-ws").close)
setUp(scn.inject(rampUsers(numSessions) during (rampPeriod.seconds))).protocols(httpProtocol)
}
class MeLoadtest extends TinodeBase {
val username = System.getProperty("username", "user0")
val password = System.getProperty("password", "user0123")
val scn = scenario("WebSocket")
.exec(ws("Connect WS").connect("/v0/channels?apikey=AQEAAAABAAD_rAp4DJh05a1HAwFT3A6K"))
.exec(session => session.set("id", "tn-" + session.userId))
.exec(session => session.set("username", username))
.exec(session => session.set("password", password))
.pause(1)
.exec(hello)
.pause(1)
.doIfOrElse({session =>
val uname = session("username").as[String]
val token = tokenCache.get(username)
token == None
}) { loginBasic } { loginToken }
.exitHereIfFailed
.exec(subMe)
.exitHereIfFailed
.exec(getSubs)
.exitHereIfFailed
.pause(1000)
.exec(ws("close-ws").close)
setUp(scn.inject(rampUsers(numSessions) during (rampPeriod.seconds))).protocols(httpProtocol)
}
class SingleTopicLoadtest extends TinodeBase {
// Input file can be set with the "accounts" java option.
// E.g. JAVA_OPTS="-Daccounts=/tmp/z.csv" gatling.sh -sf . -rsf . -rd "na" -s tinode.Loadtest
val feeder = csv(System.getProperty("accounts", "users.csv")).random
val topic = System.getProperty("topic", "TOPIC_NAME")
val scn = scenario("WebSocket")
.exec(ws("Connect WS").connect("/v0/channels?apikey=AQEAAAABAAD_rAp4DJh05a1HAwFT3A6K"))
.exec(session => session.set("id", "tn-" + session.userId))
.pause(1)
.exec(hello)
.pause(1)
.feed(feeder)
.doIfOrElse({session =>
val uname = session("username").as[String]
var token = session("token").asOption[String]
if (token == None) {
token = tokenCache.get(uname)
}
token == None
}) { loginBasic } { loginToken }
.exitHereIfFailed
.exec(subMe)
.exitHereIfFailed
.exec(getSubs)
.exitHereIfFailed
.doIf({session =>
session.attributes.contains("subs")
}) {
exec(session => session.set("sub", topic))
.exec(subTopic)
.exitHereIfFailed
.pause(0, 10)
.exec(publish)
.pause(15)
.exec(leaveTopic)
.pause(0, 3)
}
.exec(ws("close-ws").close)
setUp(scn.inject(rampUsers(numSessions) during (rampPeriod.seconds))).protocols(httpProtocol)
}
Binary file not shown.
+45
View File
@@ -0,0 +1,45 @@
%% Support module for Tinode load testing with Tsung.
%% Compile using erlc then copy resulting .beam to
%% /usr/local/lib/erlang/lib/tsung-1.7.0/ebin/
%% Alternatively you can just leave it in the current
%% directory.
-module(tinode).
-export([rand_user_secret/1, shuffle/1, cache_token/2, read_token/1]).
%% Produces a secret for use in basic login.
rand_user_secret({Pid, DynData}) ->
base64:encode_to_string(get_rand_secret()).
%% Unexported. Picks a random user from a pre-defined list.
get_rand_secret() ->
case rand:uniform(6) of
1 -> "alice:alice123";
2 -> "bob:bob123";
3 -> "carol:carol123";
4 -> "dave:dave123";
5 -> "eve:eve123";
6 -> "frank:frank123"
end.
%% Shuffles a list randomly.
shuffle(L) ->
RandomList=[{rand:uniform(), X} || X <- L],
[X || {_,X} <- lists:sort(RandomList)].
%% Reads previously cached auth token for the specified user.
read_token(Uid) ->
{ok, LogDir} = application:get_env(tsung_controller, log_dir_real),
case file:read_file(filename:join(LogDir, Uid)) of
{ok, Data} -> string:trim(Data);
{error, _} -> ""
end.
%% Saves auth token for the specified user in the log directory.
cache_token(Uid, Token) ->
{ok, LogDir} = application:get_env(tsung_controller, log_dir_real),
{ok, File} = file:open(filename:join(LogDir, Uid), [write]),
file:write(File, Token),
file:close(File),
ok.
+154
View File
@@ -0,0 +1,154 @@
package tinode
import java.util.Base64
import java.util.concurrent.ConcurrentHashMap
import scala.collection.JavaConverters._
import scala.collection._
import scala.concurrent.duration._
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class TinodeBase extends Simulation {
val httpProtocol = http
.baseUrl("http://localhost:6060")
.wsBaseUrl("ws://localhost:6060")
// Auth tokens to share between sessions.
val tokenCache : concurrent.Map[String, String] = new ConcurrentHashMap() asScala
// Total number of messages to publish to a topic.
val publishCount = Integer.getInteger("publish_count", 10).toInt
// Maximum interval between publishing messages to a topic.
val publishInterval = Integer.getInteger("publish_interval", 100).toInt
// Total number of sessions.
val numSessions = Integer.getInteger("num_sessions", 10000)
// Ramp up period (0 to numSessions) in seconds.
val rampPeriod = java.lang.Long.getLong("ramp", 300L)
val hello = exitBlockOnFail {
exec {
ws("hi").sendText(
"""{"hi":{"id":"afabb3","ver":"0.22.8","ua":"Gatling-Loadtest/1.0; gatling/1.7.0"}}"""
)
.await(15 seconds)(
ws.checkTextMessage("hi")
.matching(jsonPath("$.ctrl").find.exists)
)
}
}
val loginBasic = exitBlockOnFail {
exec { session =>
val uname = session("username").as[String]
val password = session("password").as[String]
val secret = new String(java.util.Base64.getEncoder.encode((uname + ":" + password).getBytes()))
session.set("secret", secret)
}
.exec {
ws("login").sendText(
"""{"login":{"id":"${id}-login","scheme":"basic","secret":"${secret}"}}"""
)
.await(15 seconds)(
ws.checkTextMessage("login-ctrl")
.matching(jsonPath("$.ctrl").find.exists)
.check(jsonPath("$.ctrl.params.token").saveAs("token"))
)
}
.exec { session =>
val uname = session("username").as[String]
val token = session("token").as[String]
tokenCache.put(uname, token)
session
}
}
val loginToken = exitBlockOnFail {
exec { session =>
val uname = session("username").as[String]
var token = session("token").asOption[String]
if (token == None) {
token = tokenCache.get(uname)
}
session.set("token", token.getOrElse(""))
}
.exec {
ws("login-token").sendText(
"""{"login":{"id":"${id}-login2","scheme":"token","secret":"${token}"}}"""
)
.await(15 seconds)(
ws.checkTextMessage("login-ctrl")
.matching(jsonPath("$.ctrl").find.exists)
)
}
}
val subMe = exitBlockOnFail {
exec {
ws("sub-me").sendText(
"""{"sub":{"id":"${id}-sub-me","topic":"me","get":{"what":"desc"}}}"""
)
.await(15 seconds)(
ws.checkTextMessage("sub-me-desc")
.matching(jsonPath("$.ctrl").find.exists)
.check(jsonPath("$.ctrl.code").ofType[Int].in(200 to 299))
)
}
}
val subTopic = exitBlockOnFail {
exec {
ws("sub-topic").sendText(
"""{"sub":{"id":"${id}-sub-${sub}","topic":"${sub}","get":{"what":"desc sub data del"}}}"""
)
.await(15 seconds)(
ws.checkTextMessage("sub-topic-ctrl")
.matching(jsonPath("$.ctrl").find.exists)
.check(jsonPath("$.ctrl.code").ofType[Int].in(200 to 299))
)
}
}
val publish = exitBlockOnFail {
exec {
repeat(publishCount, "i") {
exec {
ws("pub-topic").sendText(
"""{"pub":{"id":"${id}-pub-${sub}-${i}","topic":"${sub}","content":"This is a Gatling test ${i}"}}"""
)
.await(15 seconds)(
ws.checkTextMessage("pub-topic-ctrl")
.matching(jsonPath("$.ctrl").find.exists)
.check(jsonPath("$.ctrl.code").ofType[Int].in(200 to 299))
)
}
.pause(0, publishInterval)
}
}
}
val getSubs = exitBlockOnFail {
exec {
ws("get-subs").sendText(
"""{"get":{"id":"${id}-get-subs","topic":"me","what":"sub"}}"""
)
.await(15 seconds)(
ws.checkTextMessage("save-subs")
.matching(jsonPath("$.meta.sub").find.exists)
.check(jsonPath("$.meta.sub[*].topic").findAll.saveAs("subs"))
)
}
}
val leaveTopic = exitBlockOnFail {
exec {
ws("leave-topic").sendText(
"""{"leave":{"id":"${id}-leave-${sub}","topic":"${sub}"}}"""
)
.await(15 seconds)(
ws.checkTextMessage("sub-topic-ctrl")
.matching(jsonPath("$.ctrl").find.exists)
)
}
}
}
+136
View File
@@ -0,0 +1,136 @@
<?xml version="1.0"?>
<!DOCTYPE tsung SYSTEM "/usr/local/share/tsung/tsung-1.0.dtd" []>
<tsung loglevel="notice" version="1.0" dumptraffic="false">
<clients>
<client host="localhost" use_controller_vm="true" maxusers="40000" />
</clients>
<servers>
<server host="127.0.0.1" port="6060" type="tcp" />
</servers>
<load>
<arrivalphase phase="1" duration="120" unit="second">
<users maxnumber="40000" arrivalrate="50" unit="second" />
</arrivalphase>
</load>
<sessions>
<session name="websocket" probability="100" type="ts_websocket">
<setdynvars sourcetype="random_string" length="4">
<var name="baseid" />
</setdynvars>
<request subst="true">
<websocket type="connect" path="/v0/channels?apikey=AQEAAAABAAD_rAp4DJh05a1HAwFT3A6K">
<!--http_header name="X-Tinode-APIKey" value="AQEAAAABAAD_rAp4DJh05a1HAwFT3A6K"/-->
</websocket>
</request>
<request subst="true">
<websocket type="message">{"hi":{"id":"%%_baseid%%01","ver":"0.15","ua":"Tsung-Loadtest/1.0; tsung/1.7.0"}}</websocket>
</request>
<!-- Randomly pick a user. -->
<setdynvars sourcetype="eval"
code='fun({Pid, DynVars}) ->
tinode:rand_user_secret({Pid, DynVars})
end.'>
<var name="secret" />
</setdynvars>
<!-- Read auth token from the cache. -->
<setdynvars sourcetype="eval"
code='fun({Pid, DynVars}) ->
{ok, Secret} = ts_dynvars:lookup(secret, DynVars),
tinode:read_token(Secret)
end.'>
<var name="token" />
</setdynvars>
<!-- Token present. Authenticate with it. -->
<if var="token" neq="">
<request subst="true">
<match do="abort" when="nomatch">{"ctrl":.*"code":200.*}</match>
<websocket type="message">{"login":{"id":"%%_baseid%%02","scheme":"token","secret":"%%_token%%"}}</websocket>
</request>
</if>
<!-- else log in with user name and password. -->
<if var="token" eq="">
<request subst="true">
<match do="abort" when="nomatch">{"ctrl":.*"code":200.*}</match>
<dyn_variable name="token" re='^{"ctrl":.*"token":"([-A-Za-z0-9+\/=]+={0,3})".*}$'/>
<websocket type="message">{"login":{"id":"%%_baseid%%02","scheme":"basic","secret":"%%_secret%%"}}</websocket>
</request>
<!-- Save token in the cache. -->
<setdynvars sourcetype="eval"
code='fun({_, DynVars}) ->
{ok, Token} = ts_dynvars:lookup(token, DynVars),
{ok, Secret} = ts_dynvars:lookup(secret, DynVars),
tinode:cache_token(Secret, Token)
end.'>
<var name="dummy" />
</setdynvars>
</if>
<request subst="true">
<websocket type="message" frame="text">{"sub":{"id":"%%_baseid%%03","topic":"me","get":{"what":"desc"}}}</websocket>
</request>
<thinktime value="3" random="true"></thinktime>
<request subst="true">
<!--
Response looks like:
{"meta":{"id":"106691","topic":"me","ts":"2018-07-25T18:14:24.433Z","sub":[
{"updated":"2018-07-16T01:29:25.016Z","acs":{"given":47,"want":47,"mode":47},"topic":"grpOZcxbbwE7Hc","touched":"2018-07-16T17:32:12.48Z","seq":3},{"updated":"2018-07-16T01:35:25.016Z","acs":{"given":47,"want":47,"mode":47},"topic":"grpz6jDYiygVkI","touched":"2018-07-16T17:30:32.165Z","seq":22},{"updated":"2018-07-16T02:05:25.019Z","acs":{"given":47,"want":47,"mode":47},"topic":"grp8efPpvZoUQ4","touched":"2018-07-16T17:24:34.946Z","seq":1},{"updated":"2018-07-15T17:23:25.002Z","acs":{"given":31,"want":31,"mode":31},"topic":"usrlF7rSfaEx9c","touched":"2018-07-16T17:28:42.07Z","seq":9},{"updated":"2018-07-15T23:23:25.008Z","acs":{"given":16,"want":31,"mode":16},"topic":"usrddg3AFqzeEQ"},{"updated":"2018-07-15T23:53:25.009Z","acs":{"given":31,"want":31,"mode":31},"topic":"usrR6qq94MxLZ0","touched":"2018-07-16T17:30:03.93Z","seq":11},{"updated":"2018-07-15T23:56:25.01Z","acs":{"given":31,"want":31,"mode":31},"topic":"usriUhihEisbH8","touched":"2018-07-16T17:26:20.401Z","seq":9}]}}
-->
<dyn_variable name="subs" jsonpath="meta.sub[*].topic"/>
<websocket type="message" frame="text">{"get":{"id":"%%_baseid%%04","topic":"me","what":"sub"}}</websocket>
</request>
<setdynvars sourcetype="eval"
code="fun({Pid,DynVars})->
{ok,Subs}=ts_dynvars:lookup(subs,DynVars),
tinode:shuffle(Subs) end.">
<var name="shuffled_subs" />
</setdynvars>
<!-- main loop -->
<for from="1" to="100" incr="1" var="ctr">
<!-- Iterate over subscriptions -->
<foreach name="topicx" in="shuffled_subs">
<thinktime value="1" random="true"></thinktime>
<request subst="true">
<dyn_variable name="code" jsonpath="ctrl.code"/>
<websocket type="message" frame="text">{"sub":{"id":"%%_baseid%%%%_topicx%%%%_ctr%%","topic":"%%_topicx%%","get":{"what":"desc sub data del"}}}</websocket>
</request>
<if var="code" eq="200">
<for from="1" to="50" incr="1" var="counter">
<thinktime min="2" max="10" random="true"></thinktime>
<request subst="true">
<websocket type="message" frame="text">{"pub":{"id":"%%_baseid%%%%_topicx%%%%_ctr%%%%_counter%%","topic":"%%_topicx%%","content":"This is a Tsung test %%_baseid%% %%_counter%%"}}</websocket>
</request>
</for>
</if>
<request subst="true">
<websocket type="message" frame="text">{"leave":{"id":"%%_baseid%%%%_topicx%%%%_ctr%%","topic":"%%_topicx%%"}}</websocket>
</request>
</foreach>
</for>
<!-- end main loop -->
<request>
<websocket type="close"></websocket>
</request>
</session>
</sessions>
</tsung>
+7
View File
@@ -0,0 +1,7 @@
username,password
alice,alice123
bob,bob123
carol,carol123
dave,dave123
eve,eve123
frank,frank123
1 username password
1 username password
2 alice alice123
3 bob bob123
4 carol carol123
5 dave dave123
6 eve eve123
7 frank frank123