Shibuya developers

If we split the work of Shibuya into two big categories, these are the two:

  1. Frontend(UI). They are mainly based on Boostrap and Vue.js.
  2. Backend(API, controller, shibuya-agent). All of them are written in Go.

Backend

API

Shibuya follows RESTful API standard. They can be found in shibuya/api/main.go

Controller

This is where the core Shibuya functionality is. Below is the core functionaties of the controller:

  1. Engines lifecycle management with the scheduler, Kubernetes is the only scheduler we are supporting. Lifecycle management includes launch/purge/healthcheck the engines.
  2. Trigger engines and stop the tests engines. This will essentially start and stop the tests.
  3. Stream the metrics generated by the engines, such as latency, thread number. and expose them as Prometheus metrics.

Most of the above logic can be found in shibuya/controller/engine.go

Engine is an abstraction name of load generator. Shibuya is designed to supported multiple load generators since day 1. Currently, the only load generator we are supporting is Apache Jmeter.

Metrics streaming

Controller reads all the metrics from engines in Server Side Events. The content of the events can be customised as you would need to write your own parser to parse the metrics and expose them via Prometheus. For example, you can check the current Jmeter logic at shibuya/controller/jmeter.go

Since the controller is reading the events from the engine, the server side implementation also needs to be taken care of. We will discuss this part in the shibuya-agent.

shibuya-agent

shibuya-agent is a process running alongside with load generator in order to work with controller. The main responsibility of the agent is:

  1. Take the HTTP requests from controller. Such as start the test, stop the test, healthcheck requests, etc
  2. Forward the metrics generated by load generator back to the controller via SSE. The metrics have to be text based in order for streaming.

For current Jmeter implementation, you can check here shibuya/engines/jmeter/shibuya-agent.go