In this paper, through a complete example of the project, from the Golang project structure, hierarchical ideas, dependency injection, error handling, unit testing, service governance, framework selection and other aspects of the Go language project development experience.
purpose
- Provides a complete programming example of the GO language project
- Describes the specifications that Go development should follow
- The application of programming ideas in GO language is introduced
- Service governance related framework, middleware introduction
The sample project
Go – making source code project – sample
Quick start
git clone https://github.com/sdgmf/go-project-sample.git
cd go-project-sample
make docker-compose
Copy the code- Access interface: http://localhost:8080/product/1
- consul: http://localhost:8500/
- grafana: http://localhost:3000/
- jaeger: http://localhost:16686/search
- Prometheus: http://localhost:9090/graph
Package structure
Dave Chaney’s blog Five Suggestions for Setting up a Go Project discusses package design suggestions for Package and Command for golang’s package structure, There is also a community-approved package structure specification, Project-Layout. Based on the knowledge of these two articles and common Internet micro-service projects, I refined the project structure as follows.
. ├ ─ ─ API │ └ ─ ─ proto ├ ─ ─ build │ ├ ─ ─ the details │ ├ ─ ─ products │ ├ ─ ─ ratings │ └ ─ ─ reviews ├ ─ ─ CMD │ ├ ─ ─ the details │ ├ ─ ─ Products │ ├─ Ratings │ ├── Bass Exercises ─ deployments Bass Exercises ── Bass Exercises ─ app │ ├─ Details │ ├─ ├ ─ ─ controllers │ │ │ ├ ─ ─ grpcservers │ │ │ ├ ─ ─ repositorys │ │ │ └ ─ ─ services │ │ ├ ─ ─ products │ │ │ ├ ─ ─ controllers │ │ │ ├ ─ ─ grpcclients │ │ │ └ ─ ─ services │ │ ├ ─ ─ ratings │ │ │ ├ ─ ─ controllers │ │ │ ├ ─ ─ grpcservers │ │ │ ├ ─ ─ repositorys │ │ │ └ ─ ─ services │ │ └ ─ ─ reviews │ │ ├ ─ ─ controllers │ │ ├ ─ ─ grpcservers │ │ ├ ─ ─ repositorys │ │ └ ─ ─ services │ └ ─ ─ PKG │ ├ ─ ─ app │ ├ ─ ─ the config │ ├ ─ ─ consul │ ├ ─ ─ the database │ ├ ─ ─ jaeger │ ├ ─ ─log│ ├ ─ ─ models │ ├ ─ ─ transports │ │ ├ ─ ─ GRPC │ │ └ ─ ─ HTTP │ │ └ ─ ─ middlewares │ │ └ ─ ─ ginprom │ └ ─ ─ utils │ └ ─ ─ netutil ├ ─ ─ away └ ─ ─ scriptsCopy the code/cmd
With the project layout
“Main method of the project. The directory name for each application should match the name of the executable you want to have (for example, / CMD /myapp). Don’t put a lot of code in the application directory. If you think the code can be imported and used in other projects, it should exist in the/PKG directory. If your code is not reusable or you don’t want others to reuse it, put it in the/internal directory. You’ll be surprised what others will do, so be clear about your intentions! There is usually a small main function that imports and calls code from the/internal and/PKG directories, rather than anything else.
/internal/pkg
With the project layout
“Private application and library code. This is code that you don’t want someone else importing into their application or library. Put your actual application code in the /internal/app directory (e.g. /internal/app/myapp) and the code shared by these applications in the /internal/ PKG directory (e.g. /internal/ PKG /myprivlib).
The inner package is tiled.
/internal/pkg/config
Load the configuration file, or get the configuration from the configuration center and listen for configuration changes.
/internal/pkg/database
Initialize the database connection and ORM framework.
/internal/pkg/models
Structure definition.
/internal/pkg/transport
HTTP/GPC transport layer
/internal/app/products
In-app code
/internal/app/products/controllers
MVC control layer
/internal/app/products/services
Domain logic layer
/internal/app/products/repositorys
Storage layer
/internal/app/products/grpcclients
grpc client
/internal/app/details/grpcservers
grpc servers
/mocks
Mock implementations generated by SHANzhai
/api
OpenAPI/Swagger specification, JSON schema file, protocol definition file, etc.
/grafana
Generate the scripts used by the Grafana dashboard
/scripts
SQL, deployment scripts, etc
/build
Dockerfile, docker – compose
/deployment
Docker – compose/kubernetes configuration
layered
MVC, domain model and ORM are all separated into different hierarchical objects through the code of specific responsibilities. In Java, these hierarchical concepts are reflected in various frameworks (such as SSH,SSM and other commonly used framework combinations), and have already formed the default protocol. Is it still applicable to go language? The answer is yes. Martin Fowler explains the benefits of layering in His book Enterprise Application Architecture Patterns.
- Then code reuse, improve code maintainability. For example, the code of Service can be reused by HTTP and GRPC. It is also convenient to add the interface of thrift.
- The hierarchy is clear and the code is more readable.
- Unit testing, which often fails because of its reliance on persistent storage, is easier if the persistence code is extracted into separate objects.
Dependency injection
Java programmers are familiar with the idea of dependency injection and control flipping, and Spring was formally developed with dependency injection in mind. The benefit of dependency injection is decoupling, leaving the assembly of objects to the control of the container (selecting the required implementation class, whether or not a singleton is instantiated, and initialization). Dependency injection is a convenient way to implement unit testing and improve code maintainability.
Golang Dependency Injection Package: Uber dig, FX, Facebook Inject, And Google Wire Dig, FX, and Inject are implemented based on reflection, while Wire is implemented through code generation, which is explicit.
This example uses Wire to do dependency injection. Write wire.go. Wire generates code from wire.go.
// +build wireinject
package main
import (
"github.com/google/wire"
"github.com/zlgwzy/go-project-sample/cmd/app"
"github.com/zlgwzy/go-project-sample/internal/pkg/config"
"github.com/zlgwzy/go-project-sample/internal/pkg/database"
"github.com/zlgwzy/go-project-sample/internal/pkg/log"
"github.com/zlgwzy/go-project-sample/internal/pkg/services"
"github.com/zlgwzy/go-project-sample/internal/pkg/repositorys"
"github.com/zlgwzy/go-project-sample/internal/pkg/transport/http"
"github.com/zlgwzy/go-project-sample/internal/pkg/transport/grpc"
)
var providerSet = wire.NewSet(
log.ProviderSet,
config.ProviderSet,
database.ProviderSet,
services.ProviderSet,
repositorys.ProviderSet,
http.ProviderSet,
grpc.ProviderSet,
app.ProviderSet,
)
func CreateApp(cf string) (*app.App, error) {
panic(wire.Build(providerSet))
}
Copy the codeThe generated code
go get github.com/google/wire/cmd/wire
wire ./...
Copy the codeThe generated code is in wire_gen.go
// Code generated by Wire. DO NOT EDIT.
//go:generate wire
//+build ! wireinject
package main
import (
"github.com/google/wire"
"github.com/zlgwzy/go-project-sample/cmd/app"
"github.com/zlgwzy/go-project-sample/internal/pkg/config"
"github.com/zlgwzy/go-project-sample/internal/pkg/database"
"github.com/zlgwzy/go-project-sample/internal/pkg/log"
"github.com/zlgwzy/go-project-sample/internal/pkg/services"
"github.com/zlgwzy/go-project-sample/internal/pkg/repositorys"
"github.com/zlgwzy/go-project-sample/internal/pkg/transport/grpc"
"github.com/zlgwzy/go-project-sample/internal/app/proxy/grpcservers"
"github.com/zlgwzy/go-project-sample/internal/pkg/transport/http"
"github.com/zlgwzy/go-project-sample/internal/app/proxy/controllers"
)
// Injectors from wire.go:
func CreateApp(cf string) (*app.App, error) {
viper, err := config.New(cf)
iferr ! =nil {
return nil, err
}
options, err := log.NewOptions(viper)
iferr ! =nil {
return nil, err
}
logger, err := log.New(options)
iferr ! =nil {
return nil, err
}
httpOptions, err := http.NewOptions(viper)
iferr ! =nil {
return nil, err
}
databaseOptions, err := database.NewOptions(viper, logger)
iferr ! =nil {
return nil, err
}
db, err := database.New(databaseOptions)
iferr ! =nil {
return nil, err
}
productsRepository := repositorys.NewMysqlProductsRepository(logger, db)
productsService := services.NewProductService(logger, productsRepository)
productsController := controllers.NewProductsController(logger, productsService)
initControllers := controllers.CreateInitControllersFn(productsController)
engine := http.NewRouter(httpOptions, initControllers)
server, err := http.New(httpOptions, logger, engine)
iferr ! =nil {
return nil, err
}
grpcOptions, err := grpc.NewOptions(viper)
iferr ! =nil {
return nil, err
}
productsServer, err := grpcservers.NewProductsServer(logger, productsService)
iferr ! =nil {
return nil, err
}
initServers := grpcservers.CreateInitServersFn(productsServer)
grpcServer, err := grpc.New(grpcOptions, logger, initServers)
iferr ! =nil {
return nil, err
}
appApp, err := app.New(logger, server, grpcServer)
iferr ! =nil {
return nil, err
}
return appApp, nil
}
// wire.go:
var providerSet = wire.NewSet(log.ProviderSet, config.ProviderSet, database.ProviderSet, services.ProviderSet, repositorys.ProviderSet, http.ProviderSet, grpc.ProviderSet, app.ProviderSet)
Copy the codeInterface oriented programming
Polymorphism and unit tests must be better understood than explained.
Explicit programming
Golang’s development promotes the idea of explicit programming, explicit initialization, method calls, and error handling.
- Avoid using package-level global variables whenever possible.
- Try not to use init. Instead, you can call the initialization in the main function to read the code and control the initialization order.
- To return an error, use if err! = nil Explicit processing error.
- The parameters of a dependency give the caller control (the idea of controlling rollover), see dependency injection below.
Several of the big guys have discussed this, and Dr. Peter’s A Theory of Modern Go suggests that the core of the magic code is “No package level Vars; No func init “. Dave Cheny explains this in more detail in Go-without-package-Scoped-Variables.
Print log
I use more of the two logging libraries, Logrush and Zap, and PERSONALLY PREFER Zap.
Initialize logger, load log related configuration through Viper, lumberJack is responsible for log cutting.
// Options is log configration struct
type Options struct {
Filename string
MaxSize int
MaxBackups int
MaxAge int
Level string
Stdout bool
}
func NewOptions(v *viper.Viper) (*Options, error) {
var (
err error
o = new(Options)
)
if err = v.UnmarshalKey("log", o); err ! =nil {
return nil, err
}
return o, err
}
// New for init zap log library
func New(o *Options) (*zap.Logger, error) {
var (
err error
level = zap.NewAtomicLevel()
logger *zap.Logger
)
err = level.UnmarshalText([]byte(o.Level))
iferr ! =nil {
return nil, err
}
fw := zapcore.AddSync(&lumberjack.Logger{
Filename: o.Filename,
MaxSize: o.MaxSize, // megabytes
MaxBackups: o.MaxBackups,
MaxAge: o.MaxAge, // days
})
cw := zapcore.Lock(os.Stdout)
// File core uses jsonEncoder
cores := make([]zapcore.Core, 0.2)
je := zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig())
cores = append(cores, zapcore.NewCore(je, fw, level))
// Stdout Core ConsoleEncoder
if o.Stdout {
ce := zapcore.NewConsoleEncoder(zap.NewDevelopmentEncoderConfig())
cores = append(cores, zapcore.NewCore(ce, cw, level))
}
core := zapcore.NewTee(cores...)
logger = zap.New(core)
zap.ReplaceGlobals(logger)
return logger, err
}
Copy the codeLogger should be a private variable so that the object can be identified uniformly.
type Object struct {
logger *zap.Logger
}
// add a unified tag
func NewObject(logger *zap.Logger){
return &Object{
logger: logger.With(zap.String("type"."Object"))}}Copy the codeError handling
Dave Cheny’s blog Stack Traces and the Errors Package, Don’t Just Check Errors, Handle Them Gracefully.
- Use type error.
- Packaging error, recording the context of the error.
- Use the pakcage errors
- Only handle errors once, and handling errors means checking the error value and making a decision.
The error log
logger.Error("get product by id error", zap.Error(err))
Copy the code{"level":"error", "ts":1564056905.4602501, "MSG ":"get product by id error", "error":"product service get product error: get product error[id=2]: record not found", "errorVerbose":"record not found get product error[id=2] github.com/zlgwzy/go-project-sample/internal/pkg/repositorys.(*MysqlProductsRepository).Get /Users/xxx/code/go/go-project-sample/internal/pkg/repositorys/products.go:29 github.com/zlgwzy/go-project-sample/internal/pkg/services.(*DefaultProductsService).Get /Users/xxx/code/go/go-project-sample/internal/pkg/services/products.go:27 github.com/zlgwzy/go-project-sample/internal/app/proxy/controllers.(*ProductsController).Get /Users/xxx/code/go/go-project-sample/internal/app/proxy/controllers/products.go:30 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 Github.com/gin-gonic/gin.RecoveryWithWriter.func1 / Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:83 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 Engine github.com/gin-gonic/gin. (*). HandleHTTPRequest/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:389 Engine github.com/gin-gonic/gin. (*). ServeHTTP/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:351 .net/HTTP. ServerHandler. ServeHTTP/usr/local/Cellar/go / 1.12.6 / libexec/SRC/net/HTTP/server go: 2774.net/HTTP. (* conn). Serve / usr/local/Cellar/go / 1.12.6 / libexec/SRC/net/HTTP/server go: 1878 runtime. Goexit / usr/local/Cellar/go / 1.12.6 / libexec/SRC/runtime/asm_amd64. S: 1337 product service get the product error github.com/zlgwzy/go-project-sample/internal/pkg/services.(*DefaultProductsService).Get /Users/xxx/code/go/go-project-sample/internal/pkg/services/products.go:28 github.com/zlgwzy/go-project-sample/internal/app/proxy/controllers.(*ProductsController).Get /Users/xxx/code/go/go-project-sample/internal/app/proxy/controllers/products.go:30 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 Github.com/gin-gonic/gin.RecoveryWithWriter.func1 / Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:83 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 Engine github.com/gin-gonic/gin. (*). HandleHTTPRequest/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:389 Engine github.com/gin-gonic/gin. (*). ServeHTTP/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:351 .net/HTTP. ServerHandler. ServeHTTP/usr/local/Cellar/go / 1.12.6 / libexec/SRC/net/HTTP/server go: 2774.net/HTTP. (* conn). Serve / usr/local/Cellar/go / 1.12.6 / libexec/SRC/net/HTTP/server go: 1878 runtime. Goexit / usr/local/Cellar/go / 1.12.6 / libexec/SRC/runtime/asm_amd64. S: 1337 "}Copy the codeAn error was returned on the interface
Gin can be used in:
func Handler(c *gin.Context) {
err := //
c.String(http.StatusInternalServerError, "%+v", err)
}
Copy the codeThe curl http://localhost:8080/product/5 output:
rpc error: code = Unknown desc = details grpc service get detail error: detail service get detail error: get product error[id=5]: record not found get rating error github.com/sdgmf/go-project-sample/internal/app/products/services.(*DefaultProductsService).Get /Users/xxx/code/go/go-project-sample/internal/app/products/services/products.go:50 github.com/sdgmf/go-project-sample/internal/app/products/controllers.(*ProductsController).Get /Users/xxx/code/go/go-project-sample/internal/app/products/controllers/products.go:30 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 github.com/opentracing-contrib/go-gin/ginhttp.Middleware.func4 / Users/xxx/go/pkg/mod/github.com/opentracing-contrib/[email protected]/ginhttp/server.go:99 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 github.com/sdgmf/go-project-sample/internal/pkg/transports/http/middlewares/ginprom.(*GinPrometheus).Middleware.func1 /Users/xxx/code/go/go-project-sample/internal/pkg/transports/http/middlewares/ginprom/ginprom.go:105 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 github.com/gin-contrib/zap.RecoveryWithZap.func1 / Users/xxx/go/pkg/mod/github.com/gin-contrib/[email protected]/zap.go:109 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 github.com/gin-contrib/zap.Ginzap.func1 / Users/xxx/go/pkg/mod/github.com/gin-contrib/[email protected]/zap.go:32 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 Github.com/gin-gonic/gin.RecoveryWithWriter.func1 / Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:83 Github.com/gin-gonic/gin. Context (*). Next/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:124 Engine github.com/gin-gonic/gin. (*). HandleHTTPRequest/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:389 Engine github.com/gin-gonic/gin. (*). ServeHTTP/Users/xxx/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:351 .net/HTTP. ServerHandler. ServeHTTP/usr/local/Cellar/go / 1.12.6 / libexec/SRC/net/HTTP/server go: 2774.net/HTTP. (* conn). Serve / usr/local/Cellar/go / 1.12.6 / libexec/SRC/net/HTTP/server go: 1878 runtime. Goexit / usr/local/Cellar/go / 1.12.6 / libexec/SRC/runtime/asm_amd64. S: 1337Copy the codeAdd the monitor
Prometheus
As a next-generation monitoring framework, Prometheus has the following features:
- Powerful multidimensional data model:
- Time series data is distinguished by metric names and key-value pairs.
- All metrics can set arbitrary multidimensional labels.
- The data model is more arbitrary and does not need to be deliberately set up as dot-separated strings.
- Data models can be aggregated, sliced and sliced.
- Supports double – precision floating – point types, and labels can be set to full Unicode.
- Flexible and powerful query statement (PromQL) : Multiple metrics can be multiplied, added, joined, and fractional in the same query statement.
- Easy to manage: Prometheus Server is a separate binary file that works directly locally and is not dependent on distributed storage.
- Efficient: only 3.5 bytes per sampling point on average, and a Prometheus Server can handle millions of metrics.
- Using pull mode to collect time series data not only facilitates native testing but also prevents a problematic server from pushing bad metrics.
- Time series data can be pushed to Prometheus Server using the Push Gateway.
- Targets can be obtained through service discovery or static configuration.
- There are multiple visual graphical interfaces.
- Easy to scale
Go basic monitoring
import (
"github.com/opentracing-contrib/go-gin/ginhttp"
"github.com/gin-gonic/gin"
)
r := gin.New()
r.GET("/metrics", gin.WrapH(promhttp.Handler()))
Copy the codeHTTP monitoring
Create internal/PKG/transports/HTTP/middlewares ginprom/ginprom. Go
package ginprom
import (
"strconv"
"sync"
"time"
"github.com/gin-gonic/gin"
"github.com/prometheus/client_golang/prometheus"
)
const (
metricsPath = "/metrics"
faviconPath = "/favicon.ico"
)
var (
httpHistogram = prometheus.NewHistogramVec(prometheus.HistogramOpts{
Namespace: "http_server",
Name: "requests_seconds",
Help: "Histogram of response latency (seconds) of http handlers.",} []string{"method"."code"."uri"}))func init(a) {
prometheus.MustRegister(httpHistogram)
}
type handlerPath struct {
sync.Map
}
func (hp *handlerPath) get(handler string) string {
v, ok := hp.Load(handler)
if! ok {return ""
}
return v.(string)}func (hp *handlerPath) set(ri gin.RouteInfo) {
hp.Store(ri.Handler, ri.Path)
}
// GinPrometheus struct
type GinPrometheus struct {
engine *gin.Engine
ignored map[string]bool
pathMap *handlerPath
updated bool
}
// Option Specifies configurable parameters
type Option func(*GinPrometheus)
// IgnoreAdd ignored pathsfunc Ignore(path ...string) Option {
return func(gp *GinPrometheus) {
for _, p := range path {
gp.ignored[p] = true}}}// New constructor
func New(e *gin.Engine, options ... Option) *GinPrometheus {
// Validate parameters
if e == nil {
return nil
}
gp := &GinPrometheus{
engine: e,
ignored: map[string]bool{
metricsPath: true,
faviconPath: true,
},
pathMap: &handlerPath{},
}
for _, o := range options {
o(gp)
}
return gp
}
func (gp *GinPrometheus) updatePath(a) {
gp.updated = true
for _, ri := range gp.engine.Routes() {
gp.pathMap.set(ri)
}
}
// Middleware returns Middleware
func (gp *GinPrometheus) Middleware(a) gin.HandlerFunc {
return func(c *gin.Context) {
if! gp.updated { gp.updatePath() }// Filter out what you don't need
if gp.ignored[c.Request.URL.String()] == true {
c.Next()
return
}
start := time.Now()
c.Next()
httpHistogram.WithLabelValues(
c.Request.Method,
strconv.Itoa(c.Writer.Status()),
gp.pathMap.get(c.HandlerName()),
).Observe(time.Since(start).Seconds())
}
}
Copy the codeIn internal/PKG/transports HTTP/HTTP/go to add:
r.Use(ginprom.New(r).Middleware()) // Add Prometheus monitoring
Copy the codeGRPC monitoring
Add:
gs = grpc.NewServer(
grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(
grpc_prometheus.StreamServerInterceptor,
)),
grpc.UnaryInterceptor(grpc_middleware.ChainUnaryServer(
grpc_prometheus.UnaryServerInterceptor,
)),
)
Copy the codeAdd to client:
grpc_prometheus.EnableClientHandlingTimeHistogram()
o.GrpcDialOptions = append(o.GrpcDialOptions,
grpc.WithInsecure(),
grpc.WithUnaryInterceptor(grpc_middleware.ChainUnaryClient(
grpc_prometheus.UnaryClientInterceptor,
),
grpc.WithStreamInterceptor(grpc_middleware.ChainStreamClient(
grpc_prometheus.StreamClientInterceptor,
),
)
Copy the codeAdd the dashbord
The dashboard can be automatically generated and integrated into the CICD system of your own company. After the launch, the dashboard will be available. The following is how to generate the dashboard through JsonNet
- Install jsonnet
- Download graonnet – lib
Create grafana/dashboard. Jsonnet
local grafana = import 'grafonnet/grafana.libsonnet';
local dashboard = grafana.dashboard;
local row = grafana.row;
local singlestat = grafana.singlestat;
local prometheus = grafana.prometheus;
local graphPanel = grafana.graphPanel;
local template = grafana.template;
local row = grafana.row;
local app = std.extVar('app');
local baseUp() = singlestat.new(
'Number of instances',
datasource='Prometheus',
span=2,
valueName='current',
transparent=true,
).addTarget(
prometheus.target(
'sum(up{app="' + app + '"})', instant=true
)
);
local baseGrpcQPS() = singlestat.new(
'Number of grpc request per seconds',
datasource='Prometheus',
span=2,
valueName='current',
transparent=true,
).addTarget(
prometheus.target(
'sum(rate(grpc_server_handled_total{app="' + app + '",grpc_type="unary"}[1m]))',
instant=true
)
);
local baseGrpcError() = singlestat.new(
'Percentage of grpc error request',
format='percent',
datasource='Prometheus',
span=2,
valueName='current',
transparent=true,
).addTarget(
prometheus.target(
'sum(rate(grpc_server_handled_total{app="' + app + '",grpc_type="unary",grpc_code!="OK"}[1m])) /sum(rate(grpc_server_started_total{app="' + app + '",grpc_type="unary"}[1m])) * 100.0',
instant=true
)
);
local baseHttpQPS() = singlestat.new(
'Number of http request per seconds',
datasource='Prometheus',
span=2,
valueName='current',
transparent=true,
).addTarget(
prometheus.target(
'sum(rate(http_server_requests_seconds_count{app="' + app + '"}[1m]))',
instant=true
)
);
local baseHttpError() = singlestat.new(
'Percentage of http error request',
datasource='Prometheus',
format='percent',
span=2,
valueName='current',
transparent=true,
).addTarget(
prometheus.target(
'sum(rate(http_server_requests_seconds_count{app="' + app + '",code!="200"}[1m])) /sum(rate(http_server_requests_seconds_count{app="' + app + '"}[1m])) * 100.0',
instant=true
)
);
local goState(metric, description=null, format='none') = graphPanel.new(
metric,
span=6,
fill=0,
min=0,
legend_values=true,
legend_min=false,
legend_max=true,
legend_current=true,
legend_total=false,
legend_avg=false,
legend_alignAsTable=true,
legend_rightSide=true,
transparent=true,
description=description,
).addTarget(
prometheus.target(
metric + '{app="' + app + '"}',
datasource='Prometheus',
legendFormat='{{instance}}'
)
);
local grpcQPS(kind='server', groups=['grpc_code']) = graphPanel.new(
//title='grpc_' + kind + '_qps_' + std.join(',', groups),
title='Number of grpc ' + kind + ' request per seconds group by (' + std.join(',', groups) + ')',
description='Number of grpc ' + kind + ' request per seconds group by (' + std.join(',', groups) + ')',
legend_values=true,
legend_max=true,
legend_current=true,
legend_alignAsTable=true,
legend_rightSide=true,
transparent=true,
span=6,
fill=0,
min=0,
).addTarget(
prometheus.target(
'sum(rate(grpc_' + kind + '_handled_total{app="' + app + '",grpc_type="unary"}[1m])) by (' + std.join(',', groups) + ')',
datasource='Prometheus',
legendFormat='{{' + std.join('}}.{{', groups) + '}}'
)
);
local grpcErrorPercentage(kind='server', groups=['instance']) = graphPanel.new(
//title='grpc_' + kind + '_error_percentage_' + std.join(',', groups),
title='Percentage of grpc ' + kind + ' error request group by (' + std.join(',', groups) + ')',
description='Percentage of grpc ' + kind + ' error request group by (' + std.join(',', groups) + ')',
format='percent',
legend_values=true,
legend_max=true,
legend_current=true,
legend_alignAsTable=true,
legend_rightSide=true,
transparent=true,
span=6,
fill=0,
min=0,
).addTarget(
prometheus.target(
'sum(rate(grpc_'+kind+'_handled_total{app="' + app + '",grpc_type="unary",grpc_code!="OK"}[1m])) by (' + std.join(',', groups) + ')/sum(rate(grpc_'+kind+'_started_total{app="' + app + '",grpc_type="unary"}[1m])) by (' + std.join(',', groups) + ')* 100.0',
datasource='Prometheus',
legendFormat='{{' + std.join('}}.{{', groups) + '}}'
)
);
local grpcLatency(kind='server', groups=['instance'], quantile='0.99') = graphPanel.new(
title='Latency of grpc ' + kind + ' request group by (' + std.join(',', groups) + ')',
description='Latency of grpc ' + kind + ' request group by (' + std.join(',', groups) + ')',
format='ms',
legend_values=true,
legend_max=true,
legend_current=true,
legend_alignAsTable=true,
legend_rightSide=true,
transparent=true,
span=6,
fill=0,
min=0,
).addTarget(
prometheus.target(
'1000 * histogram_quantile(' + quantile + ',sum(rate(grpc_' + kind + '_handling_seconds_bucket{app="' + app + '",grpc_type="unary"}[1m])) by (' + std.join(',', groups) + ',le))',
datasource='Prometheus',
legendFormat='{{' + std.join('}}.{{', groups) + '}}'
)
);
local httpQPS(kind='server', groups=['grpc_code']) = graphPanel.new(
title='Number of http' + kind + ' request group by (' + std.join(',', groups) + ') per seconds',
description='Number of http' + kind + ' request group by (' + std.join(',', groups) + ') per seconds',
legend_values=true,
legend_max=true,
legend_current=true,
legend_alignAsTable=true,
legend_rightSide=true,
transparent=true,
span=6,
fill=0,
min=0,
).addTarget(
prometheus.target(
'sum(rate(http_server_requests_seconds_count{app="' + app + '"}[1m])) by (' + std.join(',', groups) + ')',
datasource='Prometheus',
legendFormat='{{' + std.join('}}.{{', groups) + '}}'
)
);
local httpErrorPercentage(groups=['instance']) = graphPanel.new(
//title='grpc_' + kind + '_error_percentage_' + std.join(',', groups),
title='Percentage of http error request group by (' + std.join(',', groups) + ') ',
description='Percentage of http error request group by (' + std.join(',', groups) + ')',
format='percent',
legend_values=true,
legend_max=true,
legend_current=true,
legend_alignAsTable=true,
legend_rightSide=true,
transparent=true,
span=6,
fill=0,
min=0,
).addTarget(
prometheus.target(
'sum(rate(http_server_requests_seconds_count{app="' + app + '",status!="200"}[1m])) by (' + std.join(',', groups) + ')/sum(rate(http_server_requests_seconds_count{app="' + app + '"}[1m])) by (' + std.join(',', groups) + ')* 100.0',
datasource='Prometheus',
legendFormat='{{' + std.join('}}.{{', groups) + '}}'
)
);
local httpLatency(groups=['instance'], quantile='0.99') = graphPanel.new(
title='Latency of http request group by (' + std.join(',', groups) + ')',
description='Latency of http request group by (' + std.join(',', groups) + ')',
format='ms',
legend_values=true,
legend_max=true,
legend_current=true,
legend_alignAsTable=true,
legend_rightSide=true,
transparent=true,
span=6,
fill=0,
min=0,
).addTarget(
prometheus.target(
'1000 * histogram_quantile(' + quantile + ',sum(rate(http_server_requests_seconds_bucket{app="' + app + '"}[1m])) by (' + std.join(',', groups) + ',le))',
datasource='Prometheus',
legendFormat='{{' + std.join('}}.{{', groups) + '}}'
)
);
dashboard.new(app, schemaVersion=16, tags=['go'], editable=true, uid=app)
.addPanel(row.new(title='Base', collapse=true)
.addPanel(baseUp(), gridPos={ x: 0, y: 0, w: 4, h: 10 })
.addPanel(baseGrpcQPS(), gridPos={x: 4, y: 0, w: 4, h: 10 })
.addPanel(baseGrpcError(), gridPos={x: 8, y: 0, w: 4, h: 10 })
.addPanel(baseHttpQPS(), gridPos={x: 12, y: 0, w: 4, h: 10 })
.addPanel(baseHttpError(), gridPos={x: 16, y: 0, w: 4, h: 10 })
,{ })
.addPanel(row.new(title='Go', collapse=true)
.addPanel(goState('go_goroutines', 'Number of goroutines that currently exist'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_alloc_bytes', 'Number of bytes allocated and still in use'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_alloc_bytes_total', 'Total number of bytes allocated, even if freed'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_buck_hash_sys_bytes', 'Number of bytes used by the profiling bucket hash table'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_frees_total', 'Total number of frees'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_gc_cpu_fraction', "The fraction of this program's available CPU time used by the GC since the program started."), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_gc_sys_bytes', 'Number of bytes used for garbage collection system metadata'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_heap_alloc_bytes', 'Number of heap bytes allocated and still in use'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_heap_idle_bytes', 'Number of heap bytes waiting to be used'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_heap_inuse_bytes', 'Number of heap bytes that are in use'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_heap_objects', 'Number of allocated objects'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_heap_released_bytes', 'Number of heap bytes released to OS'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_heap_sys_bytes', 'Number of heap bytes obtained from system'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_last_gc_time_seconds', 'Number of seconds since 1970 of last garbage collection'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_lookups_total', 'Total number of pointer lookups'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_mallocs_total', 'Total number of mallocs'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_mcache_inuse_bytes', 'Number of bytes in use by mcache structures'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_mcache_sys_bytes', 'Number of bytes used for mcache structures obtained from system'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_mspan_inuse_bytes', 'Number of bytes in use by mspan structures'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_mspan_sys_bytes', 'Number of bytes used for mspan structures obtained from system'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_next_gc_bytes', 'Number of heap bytes when next garbage collection will take place'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_other_sys_bytes', 'Number of bytes used for other system allocations'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_stack_inuse_bytes', 'Number of bytes in use by the stack allocator'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_stack_sys_bytes', 'Number of bytes obtained from system for stack allocator'), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(goState('go_memstats_sys_bytes', 'Number of bytes obtained from system'), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Grpc Server request rate', collapse=true)
.addPanel(grpcQPS('server', ['grpc_code']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcQPS('server', ['instance']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(grpcQPS('server', ['grpc_service']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcQPS('server', ['grpc_service', 'grpc_method']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Grpc Server request error percentage', collapse=true)
.addPanel(grpcErrorPercentage('server', ['grpc_service']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcErrorPercentage('server', ['grpc_service', 'grpc_method']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(grpcErrorPercentage('server', ['instance']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Grpc server 99%-tile Latency of requests', collapse=true)
.addPanel(grpcLatency('server', ['grpc_code'], 0.99), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('server', ['instance'], 0.99), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('server', ['grpc_service'], 0.99), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('server', ['grpc_service', 'grpc_method'], 0.99), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Grpc server 90%-tile Latency of requests', collapse=true)
.addPanel(grpcLatency('server', ['grpc_code'], 0.90), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('server', ['instance'], 0.90), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('server', ['grpc_service'], 0.90), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('server', ['grpc_service', 'grpc_method'], 0.99), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Grpc client request rate', collapse=true)
.addPanel(grpcQPS('client', ['grpc_code']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcQPS('client', ['instance']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(grpcQPS('client', ['grpc_service']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcQPS('client', ['grpc_service', 'grpc_method']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Grpc client request error percentage', collapse=true)
.addPanel(grpcErrorPercentage('client', ['grpc_service']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcErrorPercentage('client', ['grpc_service', 'grpc_method']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(grpcErrorPercentage('client', ['instance']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Grpc client 99%-tile Latency of requests', collapse=true)
.addPanel(grpcLatency('client', ['grpc_code'], 0.99), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('client', ['instance'], 0.99), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('client', ['grpc_service'], 0.99), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('client', ['grpc_service', 'grpc_method'], 0.99), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Grpc client 90%-tile Latency of requests', collapse=true)
.addPanel(grpcLatency('client', ['grpc_code'], 0.90), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('client', ['instance'], 0.90), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('client', ['grpc_service'], 0.90), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(grpcLatency('client', ['grpc_service', 'grpc_method'], 0.99), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Http server request rate', collapse=true)
.addPanel(httpQPS( ['grpc_code']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(httpQPS( ['instance']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(httpQPS( ['grpc_service']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(httpQPS( ['grpc_service', 'grpc_method']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Http server request error percentage', collapse=true)
.addPanel(httpErrorPercentage( ['instance']), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(httpErrorPercentage( ['method','uri']), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Http server 99%-tile Latency of requests', collapse=true)
.addPanel(httpLatency( ['grpc_code'], 0.99), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(httpLatency( ['instance'], 0.99), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(httpLatency( ['grpc_service'], 0.99), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(httpLatency( ['grpc_service', 'grpc_method'], 0.99), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
.addPanel(row.new(title='Http server 90%-tile Latency of requests', collapse=true)
.addPanel(httpLatency( ['grpc_code'], 0.90), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(httpLatency( ['instance'], 0.90), gridPos={ x: 12, y: 0, w: 12, h: 10 })
.addPanel(httpLatency( ['grpc_service'], 0.90), gridPos={ x: 0, y: 0, w: 12, h: 10 })
.addPanel(httpLatency( ['grpc_service', 'grpc_method'], 0.90), gridPos={ x: 12, y: 0, w: 12, h: 10 })
, {})
Copy the codeCreate grafana/dashboard – API. Jsonnet
local dash = import './dashboard.jsonnet';
{
dashboard: dash,
folderId: 0,
overwrite: false,
}
Copy the codeGenerate the JSONNET configuration
jsonnet -J ./grafana/grafonnet-lib -o ./grafana/dashboards-api/$$app-api.json --ext-str app=$$app ./grafana/dashboard-api.jsonnet ;
Copy the codeResearch grafana API
curl -X DELETE --user admin:admin -H "Content-Type: application/json" 'http://localhost:3000/api/dashboards/db/? app'
curl -x POST --user admin:admin -H "Content-Type: application/json" --data-binary "@./grafana/dashboards-api/$$app-api.json" http://localhost:3000/api/dashboards/db
Copy the codeDashboard preview
An AlerManager alarm is generated
TODO
Call chain trace
Jaeger
Jaeger is an open source implementation of Uber based on Opentracing, similar to zipkin.
Create internal/PKG/jaeger/jaeger. Go
package jaeger
import (
"github.com/google/wire"
"github.com/opentracing/opentracing-go"
"github.com/pkg/errors"
"github.com/spf13/viper"
"github.com/uber/jaeger-client-go/config"
"github.com/uber/jaeger-lib/metrics/prometheus"
"go.uber.org/zap"
)
func NewConfiguration(v *viper.Viper, logger *zap.Logger) (*config.Configuration, error) {
var (
err error
c = new(config.Configuration)
)
if err = v.UnmarshalKey("jaeger", c); err ! =nil {
return nil, errors.Wrap(err, "unmarshal jaeger configuration error")
}
logger.Info("load jaeger configuration success")
return c, nil
}
func New(c *config.Configuration) (opentracing.Tracer, error) {
metricsFactory := prometheus.New()
tracer, _, err := c.NewTracer(config.Metrics(metricsFactory))
iferr ! =nil {
return nil, errors.Wrap(err, "create jaeger tracer error")}return tracer, nil
}
var ProviderSet = wire.NewSet(New, NewConfiguration)
Copy the codeGrpc
Modified internal/PKG/transports/GRPC/server. Go
gs = grpc.NewServer(
grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(
otgrpc.OpenTracingStreamServerInterceptor(tracer),
)),
grpc.UnaryInterceptor(grpc_middleware.ChainUnaryServer(
otgrpc.OpenTracingServerInterceptor(tracer),
)),
)
Copy the codeModified internal/PKG/transports/GRPC/client. Go
conn, err := grpc.DialContext(ctx, target, grpc.WithInsecure(),
grpc.WithUnaryInterceptor(grpc_middleware.ChainUnaryClient(
otgrpc.OpenTracingClientInterceptor(tracer)),
),
grpc.WithStreamInterceptor(grpc_middleware.ChainStreamClient(
otgrpc.OpenTracingStreamClientInterceptor(tracer)),
),)
Copy the codeGin
Modify the internal/PKG/transports/HTTP/HTTP. Go
import "github.com/opentracing-contrib/go-gin/ginhttp"
r.Use(ginhttp.Middleware(tracer))
Copy the codeUnit testing
Storage Layer Testing
Add Repositorys /wire.go to create the object to be tested, injecting the appropriate dependencies according to the ProviderSet.
// +build wireinject
package repositorys
import (
"github.com/google/wire"
"github.com/sdgmf/go-project-sample/internal/pkg/config"
"github.com/sdgmf/go-project-sample/internal/pkg/database"
"github.com/sdgmf/go-project-sample/internal/pkg/log"
)
var testProviderSet = wire.NewSet(
log.ProviderSet,
config.ProviderSet,
database.ProviderSet,
ProviderSet,
)
func CreateDetailRepository(f string) (DetailsRepository, error) {
panic(wire.Build(testProviderSet))
}
Copy the codeAdd Repositorys /products_test.go to repositorys/products_test.
package repositorys
import (
"flag"
"github.com/stretchr/testify/assert"
"testing"
)
var configFile = flag.String("f"."details.yml"."set config file which viper will loading.")
func TestDetailsRepository_Get(t *testing.T) {
flag.Parse()
sto, err := CreateDetailRepository(*configFile)
iferr ! =nil {
t.Fatalf("create product Repository error,%+v", err)
}
tests := []struct {
name string
id uint64
expected bool
}{
{"id=1".1.true},
{"id=2".2.true},
{"id=3".3.true}},for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := sto.Get(test.id)
if test.expected {
assert.NoError(t, err )
}else {
assert.Error(t, err)
}
})
}
}
Copy the codeRun the test
go test -v ./internal/app/details/repositorys -f $(pwd)/configs/details.yml === RUN TestDetailsRepository_Get use config file -> /Users/xxx/code/go/go-project-sample/configs/details.yml === RUN TestDetailsRepository_Get/id=1 === RUN TestDetailsRepository_Get/id=2 === RUN TestDetailsRepository_Get/id=3 --- PASS: TestDetailsRepository_Get (0.11s) -- PASS: TestDetailsRepository_Get/ ID =1 (0.00s) -- PASS: TestDetailsRepository_Get/id = 2 (0.00 s) - PASS: TestDetailsRepository_Get/id = 3 (0.00 s) PASS ok github.com/sdgmf/go-project-sample/internal/app/details/repositorys 0.128 sCopy the codeLogical layer testing
Mock objects are generated automatically through Shanzhai.
mockery --all
Copy the codeAdd internal/app/details/services/wire. Go
// +build wireinject
package services
import (
"github.com/google/wire"
"github.com/sdgmf/go-project-sample/internal/pkg/config"
"github.com/sdgmf/go-project-sample/internal/pkg/database"
"github.com/sdgmf/go-project-sample/internal/pkg/log"
"github.com/sdgmf/go-project-sample/internal/app/details/repositorys"
)
var testProviderSet = wire.NewSet(
log.ProviderSet,
config.ProviderSet,
database.ProviderSet,
ProviderSet,
)
func CreateDetailsService(cf string, sto repositorys.DetailsRepository) (DetailsService, error) {
panic(wire.Build(testProviderSet))
}
Copy the codeThe storage layer uses the generated MockProductsRepository to define the return value of the Mock method directly in the use case.
Create services/details_test. Go
package services
import (
"flag"
"github.com/sdgmf/go-project-sample/internal/pkg/models"
"github.com/sdgmf/go-project-sample/mocks"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"testing"
)
var configFile = flag.String("f"."details.yml"."set config file which viper will loading.")
func TestDetailsRepository_Get(t *testing.T) {
flag.Parse()
sto := new(mocks.DetailsRepository)
sto.On("Get", mock.AnythingOfType("uint64")).Return(func(ID uint64) (p *models.Detail) {
return &models.Detail{
ID: ID,
}
}, func(ID uint64) error {
return nil
})
svc, err := CreateDetailsService(*configFile, sto)
iferr ! =nil {
t.Fatalf("create product serviceerror,%+v", err)
}
// Tables drive tests
tests := []struct {
name string
id uint64
expected uint64
}{
{"1 + 1".1.1},
{"2 + 3".2.2},
{"4 + 5".3.3}},for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
p, err := svc.Get(test.id)
iferr ! =nil {
t.Fatalf("product service get proudct error,%+v", err)
}
assert.Equal(t, test.expected, p.ID)
})
}
}
Copy the codeControl layer testing
Add controllers/details_test.go to test using httptest
package controllers
import (
"encoding/json"
"flag"
"fmt"
"github.com/gin-gonic/gin"
"github.com/sdgmf/go-project-sample/internal/pkg/models"
"github.com/sdgmf/go-project-sample/mocks"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"io/ioutil"
"net/http/httptest"
"testing"
)
var r *gin.Engine
var configFile = flag.String("f"."details.yml"."set config file which viper will loading.")
func setup(a) {
r = gin.New()
}
func TestDetailsController_Get(t *testing.T) {
flag.Parse()
setup()
sto := new(mocks.DetailsRepository)
sto.On("Get", mock.AnythingOfType("uint64")).Return(func(ID uint64) (p *models.Detail) {
return &models.Detail{
ID: ID,
}
}, func(ID uint64) error {
return nil
})
c, err := CreateDetailsController(*configFile, sto)
iferr ! =nil {
t.Fatalf("create product serviceerror,%+v", err)
}
r.GET("/proto/:id", c.Get)
tests := []struct {
name string
id uint64
expected uint64
}{
{"1".1.1},
{"2".2.2},
{"3".3.3}},for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
uri := fmt.Sprintf("/proto/%d", test.id)
// Construct the GET request
req := httptest.NewRequest("GET", uri, nil)
// Initialize the response
w := httptest.NewRecorder()
// Call the corresponding controller interface
r.ServeHTTP(w, req)
// Extract the response
rs := w.Result()
defer func(a) {
_ = rs.Body.Close()
}()
// Read the response body
body, _ := ioutil.ReadAll(rs.Body)
p := new(models.Detail)
err := json.Unmarshal(body, p)
iferr ! =nil {
t.Errorf("unmarshal response body error:%v", err)
}
assert.Equal(t, test.expected, p.ID)
})
}
}
Copy the codeGRPC test
The test Server
Add grpcservers/details_test. Go
package grpcservers
import (
"context"
"flag"
"github.com/sdgmf/go-project-sample/api/proto"
"github.com/sdgmf/go-project-sample/internal/pkg/models"
"github.com/sdgmf/go-project-sample/mocks"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"testing"
)
var configFile = flag.String("f"."details.yml"."set config file which viper will loading.")
func TestDetailsService_Get(t *testing.T) {
flag.Parse()
service := new(mocks.DetailsService)
service.On("Get", mock.AnythingOfType("uint64")).Return(func(ID uint64) (p *models.Detail) {
return &models.Detail{
ID: ID,
}
}, func(ID uint64) error {
return nil
})
server, err := CreateDetailsServer(*configFile, service)
iferr ! =nil {
t.Fatalf("create product server error,%+v", err)
}
// Tables drive tests
tests := []struct {
name string
id uint64
expected uint64
}{
{"1 + 1".1.1},
{"2 + 3".2.2},
{"4 + 5".3.3}},for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
req := &proto.GetDetailRequest{
Id: test.id,
}
p, err := server.Get(context.Background(), req)
iferr ! =nil {
t.Fatalf("product service get proudct error,%+v", err)
}
assert.Equal(t, test.expected, p.Id)
})
}
}
Copy the codemock grpc client
/internal/app/products/services/products_test.go:
package services
import (
"context"
"flag"
"github.com/golang/protobuf/ptypes"
"github.com/sdgmf/go-project-sample/api/proto"
"github.com/sdgmf/go-project-sample/mocks"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"google.golang.org/grpc"
"testing"
)
var configFile = flag.String("f"."products.yml"."set config file which viper will loading.")
func TestDefaultProductsService_Get(t *testing.T) {
flag.Parse()
detailsCli := new(mocks.DetailsClient)
detailsCli.On("Get", mock.Anything, mock.Anything).
Return(func(ctx context.Context, req *proto.GetDetailRequest, cos ... grpc.CallOption) *proto.Detail {
return &proto.Detail{
Id: req.Id,
CreatedTime: ptypes.TimestampNow(),
}
}, func(ctx context.Context, req *proto.GetDetailRequest, cos ... grpc.CallOption) error {
return nil
})
ratingsCli := new(mocks.RatingsClient)
ratingsCli.On("Get", context.Background(), mock.AnythingOfType("*proto.GetRatingRequest")).
Return(func(ctx context.Context, req *proto.GetRatingRequest, cos ... grpc.CallOption) *proto.Rating {
return &proto.Rating{
Id: req.ProductID,
UpdatedTime: ptypes.TimestampNow(),
}
}, func(ctx context.Context, req *proto.GetRatingRequest, cos ... grpc.CallOption) error {
return nil
})
reviewsCli := new(mocks.ReviewsClient)
reviewsCli.On("Query", context.Background(), mock.AnythingOfType("*proto.QueryReviewsRequest")).
Return(func(ctx context.Context, req *proto.QueryReviewsRequest, cos ... grpc.CallOption) *proto.QueryReviewsResponse {
return &proto.QueryReviewsResponse{
Reviews: []*proto.Review{
&proto.Review{
Id: req.ProductID,
CreatedTime: ptypes.TimestampNow(),
},
},
}
}, func(ctx context.Context, req *proto.QueryReviewsRequest, cos ... grpc.CallOption) error {
return nil
})
svc, err := CreateProductsService(*configFile, detailsCli, ratingsCli, reviewsCli)
iferr ! =nil {
t.Fatalf("create product service error,%+v", err)
}
// Tables drive tests
tests := []struct {
name string
id uint64
expected bool
}{
{"id=1".1.true}},for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := svc.Get(context.Background(), test.id)
if test.expected {
assert.NoError(t, err)
} else {
assert.Error(t, err)
}
})
}
}
Copy the codeMakefile
Write one
.PHONY: run
run:
go run ./cmd -f cmd/app.yml
.PHONY: wire
wire:
wire ./...
.PHONY: test
test:
go test -v ./... -f `pwd`/cmd/app.yml -covermode=count -coverprofile=dist/test/cover.out
.PHONY: build
build:
GOOS=linux GOARCH="amd64" go build ./cmd -o dist/sample5-linux-amd64
GOOS=darwin GOARCH="amd64" go build ./cmd -o dist/sample5-darwin-amd64
.PHONY: cover
cover:
go tool cover -html=dist/test/cover.out
.PHONY: mock
mock:
mockery --all --inpkg
.PHONY: lint
lint:
golint ./...
.PHONY: proto
proto:
protoc -I api/proto ./api/proto/products.proto --go_out=plugins=grpc:api/proto
docker: build
docker-compose -f docker/sample/docker-compose.yml up
Copy the code- Make run Runs the project
- Make Wire generates dependency injection code
- Make mock Generates a mock object
- Make test Runs unit tests
- Cover Displays the test case coverage
- Make build Builds the code
- Make Lint static code checking
- Make proto generates GRPC code
- Make Docker-compse to start all services and dependent middleware, all-in-one
Framework or library
- Gin MVC library
- Gorm ORM libraries
- Viper configuration management library
- Zap logging library
- GRPC RPC library
- Cobar Command development library
- Opentracing call chain tracing
- Monitoring the Go-Prometheus service
- Wire dependency injection
