Author: Yuanliang 360 Strange dance company engineer

This manual refers to the north, hand in hand with everyone from scratch to build a complete Go as a service of the Web application – Blog

The full application can be downloaded at [github] Yuanliang/Pharos · github

Go (Golang) is an open source language developed by Google. For more information, visit the Go website

Gin is a lightweight high-performance Web framework that supports the basic features and functions required by modern Web applications. Visit Gin’s official website for more information and documentation

React is a JavaScript framework developed by Facebook. The React website

Esbuild is a new generation of JavasScript packaging tool Esbuild website

Typescript is a free and open source programming language developed by Microsoft. It is a superset of JavaScript that extends the syntax of JavaScript. The TypeScript’s official website

PostgreSQL is the database we will use to store our data. You can find out more about PostgreSQL on the PostgreSQL website.

Relevant installation is detailed in the official website will not be repeated here.

Start Gin service

First, we’ll give our Web application a name that will serve as a server for our Blog application. I’m going to use Pharos for lighthouse

cd ~/go/src
mkdir pharos
cd pharos
Copy the code

You can download and install a dependency if you don’t already have one.

go mod download github.com/gin-gonic/gin
Copy the code

Before writing our first back-end source file, we need to create the go.mod file in the project root directory required by Golang to find and import the required dependencies. The content of this document is as follows:

Module Pharos go 1.17 require github.com/gin-gonic/gin v1.7.7Copy the code

To tidy up the go.mod file, run the following command

go mod tidy
Copy the code

Now create an entry file main.go:

Package main import ("github.com/gin-gonic/gin") func main() {create the default gin router, Router := gin.Default() // Create API routing Group API := router.group ("/ API ") {// will /hello GET GET("/hello", func(CTX *gin.Context) {ctx.JSON(200, gin.H{" MSG ": "world"}) }) } router.NoRoute(func(ctx *gin.Context) { ctx.JSON(http.StatusNotFound, Gin.H{})}) // Start listening for service requests router.run (":8080")}Copy the code

Now we can start the server with the following command:

go run main.go
Copy the code

Then you can open the browser type http://localhost:8080/api/hello to verify whether the normal boot. {” MSG “:”world”}

PNG Screen Shot 2021-12-06 at 7.57.09pm.png

You can see the access in the command line tool.

Add React

With back-end services, you can now add a front-end framework. We use esbuild-create-react-app to install the React framework

Now install React in the root directory

npx esbuild-create-react-app app
cd app
yarn start | npm run start
Copy the code

During the process, you will see the choice of language. Select Typescript.

            W  E  L  C  O  M  E      T  O         

      .d88b.  .d8888b       .d8888b 888d888 8888b.  
     d8P  Y8b 88K          d88P"    888P"      "88b 
     88888888 "Y8888b.     888      888    .d888888 
     Y8b.          X88     Y88b.    888    888  888 
      "Y8888   88888P'      "Y8888P 888    "Y888888 
                                                    
     
Hello there! esbuild create react app is a minimal replacement for create react app using a truly blazing fast esbuild bundler.
Up and running in less than 1 minute with almost zero configuration needed.
     
? To get started please choose a template 
  Javascript 
❯ Typescript
Copy the code

After the installation is complete, open the package.json file in the site and add “proxy” : http://localhost:8080. This will allow the React developed service to proxy all of our requests to the Gin back end, which will listen on port 8080.

{" name ":" site ", "version" : "1.0.0", "main" : "builder. Js", "author" : "Yuan Liang", "license" : "MIT", "proxy" : "http://localhost:8080", "scripts": { "pre-commit": "lint-staged", "lint": "eslint "src/**/*.{ts,tsx}" --max-warnings=0", "start": "node builder.js", "build": "NODE_ENV=production node builder.js"}, "dependencies": {"fs-extra": "^10.0.0", "react": "^17.0.2", "react": "^ 17.0.2"}, "devDependencies" : {" @ types/node ":" ^ 16.9.1 ", "@ types/react" : "^ 17.0.20", "@ types/react - dom" : "^ 17.0.9", "@ typescript - eslint/eslint - plugin" : "^ 4.31.0", "@ typescript - eslint/parser" : "^ 4.31.0", "chokidar" : "^ 3.5.2 esbuild", ""," ^ 0.12.26 ", "eslint" : "^ 7.32.0", "eslint - config - an" : "^ 18.2.1", "eslint - config - prettier" : "^ 8.3.0 eslint - plugin -", "import" : "^ 2.24.2", "eslint - plugin - JSX - a11y" : "^ 6.4.1", "eslint - plugin - react" : "^ 7.25.1 eslint", "- the plugin - react - hooks" : "^ 4.2.0", "husky" : "^ 7.0.2", "lint - staged" : "^ 11.1.2", "prettier" : "^ 2.4.0 server -", "reload" : "^ 0.0.3", "typescript" : "^ 4.4.3"}, "lint - staged" : {" *. + (js | JSX) ": "eslint --fix", "*.+(json|css|md)": "prettier --write" }, "husky": { "hooks": { "pre-commit": "lint-staged" } } }Copy the code

In addition, there are some bugs in the proxy in live-server. A new package (server-reload) was made, and the support of POST method was added as well as the way of proxy parameter transmission.

npm install server-reload --save-dev
Copy the code

So the builder.js parameter has also been changed.

const serverParams = { port: 8181, // Set the server port. Defaults to 8080. root: 'dist', // Set root directory that's being served. Defaults to cwd. open: false, // When false, it won't load your browser by default. cors: true, // host: // Set the address to bind to. Defaults to 0.0.0.0 or process.env.ip. proxy: {path: '/ API ', target: 'http://localhost:8080/api' } // Set proxy URLs. // ignore: 'scss,my/templates', // comma-separated string for paths to ignore // file: 'index.html' // When set, serve this file (server root relative) for every 404 (useful for single-page applications) // wait: 1000, // Waits for all changes, before reloading. Defaults to 0 sec. // mount: [['/components', './node_modules']], // Mount a directory to a route. // logLevel: 2, // 0 = errors only, 1 = some, 2 = lots // middleware: [function(req, res, next) { next();  }] // Takes an array of Connect-compatible middleware that are injected into the server middleware stack } //Copy the code

Then site files and can perform yarn start | NPM can see Go run start project service data is returned.

3. Organize the directory structure and add routes

Now let’s rearrange the directory structure to feel more formal, like an architect. Sorting code by function is a good optimal time.

We will now create a Service to hold our GO service application file and add a server folder to it to hold the server.go file as the Gin startup file. Then add a router.go to the server file as the route management file for the project.

Change the main.go file to

package main

import"pharos/services/server"

func main() {
	server.Start()
}
Copy the code

The adjusted directory structure looks like this

d90bce43-14ac-478e-be3c-69bf884944de.png

Now start the service again to see the effect 🙂

Create user object and login registration method

Next, as a blog application, the first thing to do is to do user creation login management. Now let’s make a simple User object and put it in a file to run in memory. Later we will link to the database, so that we can easily manage and store data.

The first step is to create a new directory in the Services, which we will call Store. This file will hold all the data logic of our blog. Before we link the program to the data cry, we will store the users in simple objects. In the services/ Store directory, create a new file users.go

package store

type User struct {
	Username string
	Password string
}

var Users []*User
Copy the code

Delete /hello from router.go and replace it with /signup and /signin

Package Server import (" github.com/gin-gonic/gin ") func setRouter() *gin.Engine {// Creates default gin router with Logger and Recovery middleware already attached router := gin.Default() // Enables automatic redirection if the current Route can't be matched but a // handler for the path with (without) the trailing slash exists. The router. RedirectTrailingSlash = true / / Create API route group API: = the router. The group ("/API ") {API. POST ("/signup ", SignUp) api.post (" /signin ", signin)} router.noroute (func(CTX *gin.Context) {ctx.JSON(http.statusNotFound, gin.H{}) }) return router }Copy the code

SignUp and signIn are missing. We put them in services/server/user.go

Package Server import (" NET/HTTP "" pharos/services/store" "github.com/gin-gonic/gin") func signUp(CTX *gin.Context) { user := new(store.User) if err := ctx.Bind(user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" err ": Err.Error()}) return} store.users = append(store.users, user) ctx.JSON(http.statusok, gin.H{" MSG ": "Signed up successfully.", "JWT" : "123456789",})} func signIn(CTX *gin.Context) {user := new(store.user) if err := ctx.bind (user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" err ": err.Error()}) return } for _, u := range store.Users { if u.Username == user.Username && u.Password == user.Password { ctx.JSON(http.StatusOK, Gin.H{" MSG ":" Signed in successfully. ", "JWT" : "123456789",}) return}} ctx.AbortWithStatusJSON(http.statusUnauthorized, gin.H{" err ":" Sign in failed. "})}Copy the code

What this code does is. We create a new User type variable and store the User variable in the front end. Then we call bind() to bind the data to the User variable. If the bind fails, we immediately set the error code and error message and return it from the current function. If there are no errors, we set the status of the response code to OK and return JWT for authentication. You don’t have to worry about what JWT is for now, more on that later, and you probably don’t have to do your own searching.

Now you can restart the service and invoke the status of the two routesPOSTRequests require special tools to debug, or continue without debugging.

The React front end page can be downloaded directly from Github

With the front page you can use the form to register, login users. In the app/SRC/components/Auth/AuthForm. For we add a form submission of TSX submitHandler () method. Then click Submit to see the submitted data.

We also need to add a front – and back-side validation rule to the form, which is mainly to add server-side validation. Open the services/store/users.go file and change it to something like this.

type User struct {
  Username string`binding:"required,min=5,max=30"`
  Password string`binding:"required,min=7,max=32"`
}
Copy the code

This accepts the fields that are taken over and the rules for the fields Go here to find the go-related validation rules go-playground/validator validates supported fields click here

You can now enter a user name and password from the registration page (to restart the Gin service). If you do not comply with the rules, a server error will be returned.

Add a database

So far our application runs OK, we can create users and log in, but as soon as we restart the Gin service, our user data will be lost because it is only running in memory. A complete application needs a database for data storage, we save all the data to the database. The PostgreSQL installation details are not covered. The PostgreSQL installation steps are provided on the website. For now, we assume that you have installed and configured PostgreSQL. First we create a database. If PostgreSQL is not running, we need to start it and log in using the default account Postgres.

sudo service postgresql start
sudo -u postgres psql
Copy the code

After the link is successful, the database can be created

CREATE DATABASE pharos;
Copy the code

Database communication we use go-PG module. You can run the go get github.com/go-pg/pg/v10 command to install it. Now we will add a new directory file database to the Services folder and a new file database.go

package database

import (
  "github.com/go-pg/pg/v10"
)

func NewDBOptions() *pg.Options {
  return &pg.Options{
    Addr:     "localhost:5432",
    Database: "pharos",
    User:     "postgres",
    Password: "postgres",
  }
}
Copy the code

Then add a connection to the database in services/store/store.go.

Package Store import (" log "" github.com/go-pg/pg/v10") // Database connector var db * pg.db func SetDBConnection(dbOpts) * pg.options) {if dbOpts == nil {log.panicln (" DB Options can't be nil ")} else {DB = pg.connect (dbOpts)}} func GetDBConnection() *pg.DB { return db }Copy the code

Create a variable db, then create two methods Get and Set database connection methods and then Connect to the database via pg.connect. Then we went back to the services/server/server. Go increase database connection access after the launch

Package Server import (" pharos/services/database "" pharos/services/store") func Start() { store.SetDBConnection(database.NewDBOptions()) router := setRouter() // Start listening and serving requests The router. The Run (" : "8080)}Copy the code

After linking to the database, the user’s information can be saved to the database, and you can verify that the login user name and password match. Services /store/users.go Adds authentication related capabilities.

Package store import "errors" type User struct {ID int Username string 'binding:" Required,min=5, Max =30"' Password string`binding:"required,min=7,max=32"` } func AddUser(user *User) error { _, Err := db.model (user).returning (" * ").insert () if err! = nil { return err } returnnil } func Authenticate(username, password string) (*User, Error) {user := new(user) if err := db.model (user).Where(" username =? , username).Select(); err ! = nil { returnnil, err } if password ! = user.Password {returnnil, errors.New(" Password not valid. ")} return user, nil}Copy the code

The corresponding services/server/user.go also needs to be modified.

Package Server import (" NET/HTTP "" pharos/services/store" "github.com/gin-gonic/gin") func signUp(CTX *gin.Context) { user := new(store.User) if err := ctx.Bind(user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": err.Error()}) return } if err := store.AddUser(user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": Err.error ()}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Signed up successfully. ", "JWT" : "123456789",})} func signIn(CTX *gin.Context) {user := new(store.user) if err := ctx.bind (user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": err.Error()}) return } user, err := store.Authenticate(user.Username, user.Password) if err ! = nil {ctx.AbortWithStatusJSON(http.statusunauthorized, gin.H{" error ": "}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Signed in successfully. ", "JWT" : "123456789",})}Copy the code

After the database connection is created, we also need to create tables to store the data. We used the Go-PG/Migrations module for data migration. You can find the installation and use guide on Github without going into details here. We then create a folder migrations in the root directory and create a file main.go inside

package main import ( "flag" "fmt" "os" "pharos/services/database" "pharos/services/store" "github.com/go-pg/migrations/v8" ) const usageText = `This program runs command on the db. Supported commands are: - init - creates version info table in the database - up - runs all available migrations. - up [target] - runs available  migrations up to the target one. - down - reverts last migration. - reset - reverts all migrations. - version - prints current db version. - set_version [version] - sets db version without running migrations. Usage: go run *.go <command> [args] ` func main() { flag.Usage = usage flag.Parse() store.SetDBConnection(database.NewDBOptions()) db := store.GetDBConnection() oldVersion, newVersion, err := migrations.Run(db, flag.Args()...) if err ! = nil { exitf(err.Error()) } if newVersion ! = oldVersion { fmt.Printf("migrated from version %d to %d\n", oldVersion, newVersion) } else { fmt.Printf("version is %d\n", oldVersion) } } func usage() { fmt.Print(usageText) flag.PrintDefaults() os.Exit(2) } func errorf(s string, args ... interface{}) { fmt.Fprintf(os.Stderr, s+"\n", args...) } func exitf(s string, args ... interface{}) { errorf(s, args...) os.Exit(1) }Copy the code

Add a new file 1_addUserStables. Go which is similar to the official example. We can use SetDBConnection() and GetDBConnection() to define functions in the datastore package. This is the main entry point for running a data migration.

Package the main import (" FMT "" github.com/go-pg/migrations/v8") func init () {migrations. MustRegisterTx (func (db DB) error {FMT.Println(" Creating table users... ) _, err := db.Exec(`CREATE TABLE users( id SERIAL PRIMARY KEY, username TEXT NOT NULL UNIQUE, password TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), modified_at TIMESTAMPTZ NOT NULL DEFAULT NOW() )`) return err }, Func (db migrations.db) error {FMT.Println(" dropping table users... ) _, err := db.Exec(`DROP TABLE users`) return err }) }Copy the code

Then go to the Migrations folder to execute the commands

cd migrations/
go run *.go init
go run *.go up
Copy the code

We will create a Users User table for the table. We have added created_AT and modified_AT columns to the database, so we also need to add them to the User data structure definition in services/store/users.go

type User struct {
  ID         int
  Username   string`binding:"required,min=5,max=30"`
  Password   string`binding:"required,min=7,max=32"`
  CreatedAt  time.Time
  ModifiedAt time.Time
}
Copy the code

Now try to create a new account and let’s go to the database and see that the account has been stored in the database. You can also create a migration executable

cd migrations/
go build -o migrations *.go
Copy the code

And run it

cd migrations/
go build -o migrations *.go
Copy the code

There is a problem here. In the Users table we store user passwords in plain text. This is not secure, so we should generate passwords with a random seed.

type User struct {
  ID             int
  Username       string`binding:"required,min=5,max=30"`
  Password       string`pg:"-" binding:"required,min=7,max=32"`
  HashedPassword []byte`json:"-"`
  Salt           []byte`json:"-"`
  CreatedAt      time.Time
  ModifiedAt     time.Time
}
Copy the code

Modifying the Migration file

Package the main import (" FMT "" github.com/go-pg/migrations/v8") func init () {migrations. MustRegisterTx (func (db DB) error {FMT.Println(" Creating table users... ) _, err := db.Exec(`CREATE TABLE users( id SERIAL PRIMARY KEY, username TEXT NOT NULL UNIQUE, hashed_password BYTEA NOT NULL, salt BYTEA NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP, modified_at TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP )`) return err }, Func (db migrations.db) error {FMT.Println(" dropping table users... ) _, err := db.Exec(`DROP TABLE users`) return err }) }Copy the code

Now modify services/store/users.go again

package store import ( "crypto/rand" "time" "golang.org/x/crypto/bcrypt" ) type User struct { ID int Username string`binding:"required,min=5,max=30"` Password string`pg:"-" binding:"required,min=7,max=32"` HashedPassword []byte`json:"-"` Salt []byte`json:"-"` CreatedAt time.Time ModifiedAt time.Time } func AddUser(user *User) error { salt,  err := GenerateSalt() if err ! Nil {return err} toHash := append([]byte(user.password), salt...) hashedPassword, err := bcrypt.GenerateFromPassword(toHash, bcrypt.DefaultCost) if err ! = nil { return err } user.Salt = salt user.HashedPassword = hashedPassword _, Err = db.model (user).returning (" * ").insert () if err! = nil { return err } return err } func Authenticate(username, password string) (*User, Error) {user := new(user) if err := db.model (user).Where(" username =? , username).Select(); err ! = nil {returnnil, err} Salted := Append ([]byte(password), user.salt...) if err := bcrypt.CompareHashAndPassword(user.HashedPassword, salted); err ! = nil { returnnil, err } return user, nil } func GenerateSalt() ([]byte, error) { salt := make([]byte, 16) if _, err := rand.Read(salt); err ! = nil { returnnil, err } return salt, nil }Copy the code

Update the database again

cd migrations/
go run *.go reset
go run *.go up
Copy the code

Re-access the page in the browser and create an account to see that the encrypted password has been updated to the database.

Add configuration files and startup scripts

Currently we hardcoded server addresses, ports, etc., as well as database-related options. This is not an elegant solution, so we will create an environment variable file.env from which to read the relevant configuration. First create a services/conf folder and include the conf.go file in it

package conf import ( "log" "os" "strconv" ) const ( hostKey = "PHAROS_HOST" portKey = "PHAROS_PORT" dbHostKey = "PHAROS_DB_HOST" dbPortKey = "PHAROS_DB_PORT" dbNameKey = "PHAROS_DB_NAME" dbUserKey = "PHAROS_DB_USER" dbPasswordKey = "PHAROS_DB_PASSWORD" ) type Config struct { Host string Port string DbHost string DbPort string DbName string DbUser string DbPassword string } func NewConfig() Config { host, ok := os.LookupEnv(hostKey) if ! ok || host == "" { logAndPanic(hostKey) } port, ok := os.LookupEnv(portKey) if ! Ok | | port = = "{" if _, err: = strconv. Atoi (port); err ! = nil { logAndPanic(portKey) } } dbHost, ok := os.LookupEnv(dbHostKey) if ! Ok | | dbHost = = "" {logAndPanic (dbHostKey)} dbPort, ok: = OS. LookupEnv (dbPortKey) if! Ok | | dbPort = = "{" if _, err: = strconv. Atoi (dbPort); err ! = nil { logAndPanic(dbPortKey) } } dbName, ok := os.LookupEnv(dbNameKey) if ! Ok | | dbName = = "" {logAndPanic (dbNameKey)} dbUser, ok: = OS. LookupEnv (dbUserKey) if! Ok | | dbUser = = "" {logAndPanic (dbUserKey)} dbPassword, ok: = OS. LookupEnv (dbPasswordKey) if! Ok | | dbPassword = = "" {logAndPanic (dbPasswordKey)} return Config {Host: the Host, Port, the Port, DbHost: DbHost, DbPort: dbPort, DbName: dbName, DbUser: dbUser, DbPassword: DbPassword,}} func logAndPanic(envVar string) {log.Println(" ENV variable not set or value not valid: "Panic, envVar) (envVar)}Copy the code

Then modify the code to reference the configuration logic accordingly.

First change services/database/database. Go file

Package database import (" pharos/services/conf "" github.com/go-pg/pg/v10") func NewDBOptions(CFG conf.config) *pg.Options {return &pg.Options{Addr: cfg.DbHost + ":" + cfg.DbPort, Database: cfg.DbName, User: cfg.DbUser, Password: cfg.DbPassword, } }Copy the code

Services/server/server. Go and modify accordingly

Package Server import (Pharos /services/conf, pharos/services/database, pharos/services/ Store) func Start(CFG) conf.Config) { store.SetDBConnection(database.NewDBOptions(cfg)) router := setRouter() // Start listening and serving Requests the router. The Run (" : "8080)}Copy the code

The main. Go file

Package main import (" pharos/services/conf "" pharos/services/server") func main() {server.start (conf.newconfig ())}Copy the code

One more change needs to be made in the migrations/main.go file. Simply import the pharos/services/conf package and change lines.

store.SetDBConnection(database.NewDBOptions())
Copy the code

store.SetDBConnection(database.NewDBOptions(conf.NewConfig()))
Enter fullscreen mode
Copy the code

We are now ready to read the ENV variable required for the configuration. But there’s one thing missing. We need to feed these values to ENV. To do this, let’s create a new file in the root project directory named.env:

Export PHAROS_HOST=0.0.0.0 export PHAROS_PORT=8080 export PHAROS_DB_HOST=localhost export PHAROS_DB_PORT=5432 export PHAROS_DB_NAME=pharos export PHAROS_DB_USER=postgres export PHAROS_DB_PASSWORD=postgresCopy the code

Context environment variables need to be changed by executing source. env.

source .env
go run main.go
Copy the code

Develop the Cli for deployment

We now have.env, but it would be silly to do a source operation at the start of each project to change the environment. For elegance we also need to create a series of shell scripts, as well as support for development and deployment. We first create this directory in services/ CLI and then create a cli.go file.

Package CLI import (" flag "" FMT" "OS") func usage() {FMT.Print(' This program runs Pharos Backend server. usage: pharos [arguments] Supported arguments: PrintDefaults() os.exit (1)} func Parse() {flag.usage = Usage env := flag.string (" env ", "dev", `Sets run environment. Possible values are "dev" and "prod"`) flag.Parse() fmt.Println(*env) }Copy the code

Then modify the main.go file to add the reference

package main

import (
  "pharos/services/cli"
  "pharos/services/conf"
  "pharos/services/server"
)

func main() {
  cli.Parse()
  server.Start(conf.NewConfig())
}
Copy the code

Now you’re ready to write the scripts to deploy and stop our application. We’ll create a folder called scripts and add the first script to it. deploy.sh

#! /bin/bash # default ENV is dev env=dev whiletest$# -gt 0; Do case "$1" in -env) shift iftest$# -gt 0; then env=$1 fi # shift ;; *) break ;; esac done cd .. /.. /pharos source .env go build -o pharos/pharos pharos/main.go pharos -env $env &Copy the code

In the script above we first set the environment to env=dev to be the development environment. After that we will pass parameters to the script, passing them if we find them. After setting the env variable, we switch to the root directory of the project, get the env variable, and then create a folder CMD where we can put the main.go file from the root directory. Run go build =o CMD /pharos/pharos CMD /pharos/main.go at this point we will create an executable that we will use to start our service. To build the application, we start the server with CMD /pharos/pharos -env $env &, which passes the value of the env variable to our server as the -env flag.

We’ll also create a simple script stop.sh under scripts/.

#! /bin/bash

kill $(pidof pharos)
Copy the code

This script will find the process ID of our Pharos and have control over ending the process.

Before using the script, we will modify the relevant permissions.

chmod +x deploy.sh
chmod +x stop.sh
Copy the code

Now that we can control the start and end of the service, scripts/ can be executed

./deploy.sh
./stop.sh
Copy the code

7. Add logging

Logging is also a very important part of most Web applications, because we often want to know what requests are coming in and, more importantly, if there are any unexpected errors. So, as you might have guessed, this section introduces logging, and I’ll show you how to set it up, and how to separate logging in your development and production environments. We will now use the -env flag added in the previous section.

For logging, we will use the zerolog module, you can run through the go get github.com/rs/zerolog/log for this module.

Now let’s create another directory services/logging and create a logging.go file in it.

package logging import ( "fmt" "io" "io/ioutil" "os" "path/filepath" "strings" "time" "github.com/gin-gonic/gin" "github.com/rs/zerolog" "github.com/rs/zerolog/log" ) const ( logsDir = "logs" logName = "gin_production.log" ) var logFilePath = filepath.Join(logsDir, logName) func SetGinLogToFile() { gin.SetMode(gin.ReleaseMode) logFile, err := os.Create(logFilePath) if err ! = nil { log.Panic().Err(err).Msg("Error opening Gin log file") } gin.DefaultWriter = io.MultiWriter(logFile) } func ConfigureLogger(env string) { zerolog.SetGlobalLevel(zerolog.DebugLevel) switch env { case"dev": stdOutWriter := zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: "15:04:05.000"} logger := zerolog.new (stdOutWriter).with ().timestamp ().logger () log.logger = logger case"prod": createLogDir() backupLastLog() logFile := openLogFile() logFileWriter := zerolog.ConsoleWriter{Out: logFile, NoColor: true, TimeFormat: "15:04:05.000"} logger := zerolog.new (logFileWriter).with ().timestamp ().logger () log.logger = logger default: fmt.Printf("Env not valid: %s\n", env) os.Exit(2) } } func createLogDir() { if err := os.Mkdir(logsDir, 0744); err ! = nil && ! os.IsExist(err) { log.Fatal().Err(err).Msg("Unable to create logs directory.") } } func backupLastLog() { timeStamp := time.Now().Format("20060201_15_04_05") base := strings.TrimSuffix(logName, filepath.Ext(logName)) bkpLogName := base + "_" + timeStamp + "." + filepath.Ext(logName) bkpLogPath := filepath.Join(logsDir, bkpLogName) logFile, err := ioutil.ReadFile(logFilePath) if err ! = nil {if os.isnotexist (err) {return} log.panic ().err (err).Msg(" Error reading log file for backup ")} if err = ioutil.WriteFile(bkpLogPath, logFile, 0644); err ! = nil {log.panic ().err (Err).msg (" Error writing backup log file ")}} func openLogFile() * os.file {logFile, err := os.OpenFile(logFilePath, os.O_WRONLY|os.O_CREATE, 0644) if err ! = nil {log.panic ().err (Err).msg (" Error while opening log file ")} return logFile} func curentDir() string {path, err := os.Executable() if err ! = nil {log.panic ().err (Err).msg (" Can't get current directory. ")} return filepath.Dir(path)}Copy the code

We can then update services/cli/cli.go to configure logging for the environment, rather than just printing it

Package CLI import ("flag" "FMT" "OS" "pharos/services/logging") func Usage () {FMT.Print(' This program runs pharos backend server. Usage: pharos [arguments] Supported arguments: PrintDefaults() os.exit (1)} func Parse() {flag.usage = Usage env := flag.string (" env ", "dev", `Sets run environment. Possible values are "dev" and "prod"`) flag.Parse() logging.ConfigureLogger(*env) if *env == "Prod" {logging.setginlogtofile ()}}Copy the code

This may seem like a lot of code, but it’s very simple. First we configure our logs according to the environment. If env is dev, we log everything to stdout, while for prod, we log in to the file. When we log in files, we first create log directories as needed and back up previous logs, so we have new logs every time the server starts. Of course, you can create different logic for log rotation to better suit your needs. The other thing we need to do in this case is to tell Gin to run in publish mode, which will reduce unnecessary and intrusive output, and then also set the default Gin Writer to write to the log file. You could also do this in a Prod Case block, but since we actually have two different loggers (Gin’s additional logger and our Zerolog logger), I prefer to keep the two parts of the code separate. It’s just a personal preference, you can do it your way. With this collection, we can now start logging some errors. For example, let’s update the logAndPanic() function in services/conf/conf.go:

Func logAndPanic(envVar string) {log.panic ().str (" envVar ", envVar).msg (" ENV variable not set or value not valid ")}Copy the code

We can log whether an error occurred when generating the key in services/store/users.go.

func GenerateSalt() ([]byte, error) { salt := make([]byte, 16) if _, err := rand.Read(salt); err ! Error().err (Err).msg (" Unable to create salt ") returnnil, Err} return salt, nil}Copy the code

Eight, JWT authentication

Authentication is one of the most important parts of almost every Web application. We must ensure that each user can only create, read, update, and delete data that he or she is authorized to. To do this, we will use JWT (JSON Web key). Fortunately, there are various Golang modules dedicated to this. One that will be used in this guide can be found in the GitHub repository. The latest version is v3, can run through the go get github.com/cristalhq/jwt/v3 to install.

Since we will need the key to generate and validate the token, let’s add export PHAROS_JWT_SECRET=jwtSecret123 to our.env file. Of course, in production you’ll want to use some randomly generated long strings. What we should do next is add new variables to services/conf/conf.go. We will add the constant jwtSecretKey = “PHAROS_JWT_SECRET” with the rest of our constants, and then add a new field JwtSecret of type string to the configuration structure. Now we can read the new env variable and add it to the NewConfig() function:

const ( hostKey = "PHAROS_HOST" portKey = "PHAROS_PORT" dbHostKey = "PHAROS_DB_HOST" dbPortKey = "PHAROS_DB_PORT" dbNameKey = "PHAROS_DB_NAME" dbUserKey = "PHAROS_DB_USER" dbPasswordKey = "PHAROS_DB_PASSWORD" jwtSecretKey = "PHAROS_JWT_SECRET" ) type Config struct { Host string Port string DbHost string DbPort string DbName string DbUser string DbPassword string JwtSecret string } func NewConfig() Config { host, ok := os.LookupEnv(hostKey) if ! ok || host == "" { logAndPanic(hostKey) } port, ok := os.LookupEnv(portKey) if ! ok || port == "" { if _, err := strconv.Atoi(port); err ! = nil { logAndPanic(portKey) } } dbHost, ok := os.LookupEnv(dbHostKey) if ! ok || dbHost == "" { logAndPanic(dbHostKey) } dbPort, ok := os.LookupEnv(dbPortKey) if ! ok || dbPort == "" { if _, err := strconv.Atoi(dbPort); err ! = nil { logAndPanic(dbPortKey) } } dbName, ok := os.LookupEnv(dbNameKey) if ! ok || dbName == "" { logAndPanic(dbNameKey) } dbUser, ok := os.LookupEnv(dbUserKey) if ! Ok | | dbUser = = "" {logAndPanic (dbUserKey)} dbPassword, ok: = OS. LookupEnv (dbPasswordKey) if! Ok | | dbPassword = = "" {logAndPanic (dbPasswordKey)} jwtSecret, ok: = OS. LookupEnv (jwtSecretKey) if! Ok | | jwtSecret = = "" {logAndPanic (jwtSecretKey)} return Config {Host: the Host, Port, the Port, DbHost: DbHost, DbPort: dbPort, DbName: dbName, DbUser: dbUser, DbPassword: dbPassword, JwtSecret: jwtSecret, } }Copy the code

We can create a new file services/server/jwt.go

Package server import (" pharos/services/conf "github.com/cristalhq/jwt/v3" github.com/rs/zerolog/log ") var ( jwtSigner jwt.Signer jwtVerifier jwt.Verifier ) func jwtSetup(conf conf.Config) { var err error key := []byte(conf.JwtSecret) jwtSigner, err = jwt.NewSignerHS(jwt.HS256, key) if err ! = nil {log.panic ().err (Err).msg (" Error creating JWT signer ")} jwtVerifier, Err = jwt.newverifierhs (jwt.hs256, key) if err ! = nil {log.panic ().err (Err).msg (" Error creating JWT verifier ")}}Copy the code

The jwtSetup() function creates only signers and verifiers that will be used later for authentication. Now we can start the server from the services/server/server/go call this function:

Package Server import (Pharos /services/conf, pharos/services/database, pharos/services/ Store) func Start(CFG) conf.Config) { jwtSetup(cfg) store.SetDBConnection(database.NewDBOptions(cfg)) router := setRouter() // Start listening And serving Requests router.run (" :8080 ")}Copy the code

To generate the key, we will create a function in services/server/jwt.go:

func generateJWT(user *store.User) string { claims := &jwt.RegisteredClaims{ ID: fmt.Sprint(user.ID), ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour * 24 * 7)), } builder := jwt.NewBuilder(jwtSigner) token, err := builder.Build(claims) if err ! = nil {log.panic ().err (Err).msg (" Error building JWT ")} return token.string ()}Copy the code

Then we’ll call it from services/server/user.go, instead of the hard-coded string we’ve used so far for testing purposes:

Package Server import (" NET/HTTP "" pharos/services/store" "github.com/gin-gonic/gin") func signUp(CTX *gin.Context) { user := new(store.User) if err := ctx.Bind(user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": err.Error()}) return } if err := store.AddUser(user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": Err.error ()}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Signed up successfully. ", "JWT" : generateJWT(user), }) } func signIn(ctx *gin.Context) { user := new(store.User) if err := ctx.Bind(user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": err.Error()}) return } user, err := store.Authenticate(user.Username, user.Password) if err ! = nil {ctx.AbortWithStatusJSON(http.statusunauthorized, gin.H{" error ": "}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Signed in successfully. ", "JWT" : generateJWT(user), }) }Copy the code

Let’s test this either by registering or through our front-end login. Open a browser development tool and check the login or registration response. You can see that our back end now generates random JWT:

BF9C0E23-0297-4038-99CC-8F90DDB6EF8F.png

The token is now created in signIn and signUp handlers, which means we can validate it for all secure routes. To do this, we will first implement the verifyJWT() function in services/server/jwt.go. This function will take the token as a string, verify its signature, extract the ID from the declaration, and if all goes well, the user ID will be returned as an int:

func verifyJWT(tokenStr string) (int, error) { token, err := jwt.Parse([]byte(tokenStr)) if err ! Error().err (Err).str (" tokenStr ", tokenStr).msg (" Error parsing JWT "). err } if err := jwtVerifier.Verify(token.Payload(), token.Signature()); err ! Nil {log.error ().err (Err).msg (" Error verifying token ") return0, err } var claims jwt.StandardClaims if err := json.Unmarshal(token.RawClaims(), &claims); err ! = nil {log.error ().err (Err).msg (" Error unmarshalling JWT claims ") return0, err } if notExpired := claims.IsValidAt(time.Now()); ! NotExpired {return0, errors.New(" Token expired. ")} id, err := strconv.Atoi(claims. Id) if err! = nil {log.error ().err (Err).str (" claims.ID ", claims.ID).Msg(" Error converting Claims ID to number ") return0, Errors. New(" ID in token is not valid ")} return ID, err}Copy the code

Now that the generation and validation functions are complete, we are almost ready to write Gin middleware for authorization. Before we do that, we’ll add functions to get users from the database based on their IDS. In services/store/users.go, add the function:

func FetchUser(id int) (*User, Error) {user := new(user) user.id = ID err := DB.model (user).returning (" * ").select () if err! = nil {log.error ().err (Err).Msg(" Error ") returnnil, Err} return user, nil}Copy the code

You can now create a new file services/server/middleware. Go

Package Server import (NET/HTTP, Pharos /services/ Store, strings, github.com/gin-gonic/gin) FunC authorization(CTX) *gin.Context) {authHeader := ctx.getheader (" Authorization ") if authHeader == "" { CTX. AbortWithStatusJSON (HTTP StatusUnauthorized, gin. H {" error ": "Authorization header missing."}) return} headerParts := strings.split (authHeader, "") iflen(headerParts)! = 2 {ctx.AbortWithStatusJSON(http.statusunauthorized, gin.H{" error ": Authorization Header format is not valid.}) return} if headerParts[0]! = "Bearer" {ctx.abortwithStatusjson (http.statusunauthorized, gin.H{" error ": "Authorization header is missing bearer part."}) return} userID, err := verifyJWT(headerParts[1]) if ERR! = nil {ctx.AbortWithStatusJSON(http.statusunauthorized, gin.H{" error ": err.Error()}) return } user, err := store.FetchUser(userID) if err ! = nil {ctx.AbortWithStatusJSON(http.statusunauthorized, gin.H{" error ": Err.Error()}) return} ctx.set (" user ", user) ctx.next ()}Copy the code

Authorization middleware extracts the token from the authorization header. It first checks to see if the header exists and is in a valid format, and then calls the verifyJWT() function. If JWT validates, the user ID is returned. Gets the user with this ID from the database and sets it to the current user for this context. Getting the current user from the context is something we often need, so let’s extract it into an auxiliary function:

Func currentUser(CTX *gin.Context) (* store.user, error) {var err error _user, exists := ctx.get (" User ") if! Exists {err = errors.New(" Current context user not set ") log.error ().err (err).msg (" ") returnnil, err} user, ok := _user.(*store.User) if ! Ok {err = errors.New(" Context user is not valid type ") log.error ().err (err).msg (" ") returnnil, err} return user, nil}Copy the code

First, we check to see if a user is set up for this context. If not, an error is returned. Since ctx.get () returns the interface, we must check if value is of type * store.user. If not, an error is returned. When both checks pass, the current user is returned from the context.

Ix. Add Posting function

With authentication in place, it’s time to start using it. We need authentication to create, read, update, and delete users’ blog posts. Let’s start by adding a new database migration, which will create the required data tables with columns. Create a new migration file migrations/ 2_addPoststables. Go: ·

Package the main import (" FMT "" github.com/go-pg/migrations/v8") func init () {migrations. MustRegisterTx (func (db DB) error {FMT.Println(" Creating table posts... ) _, err := db.Exec(`CREATE TABLE posts( id SERIAL PRIMARY KEY, title TEXT NOT NULL, content TEXT NOT NULL, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), modified_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), user_id INT REFERENCES users ON DELETE CASCADE )`) return err }, Func (db migrations.db) error {ftt. Println(" dropping table posts... ) _, err := db.Exec(`DROP TABLE posts`) return err }) }Copy the code

Then run Migrations

cd migrations/
go run *.go up
Copy the code

Now let’s create a structure to hold the post data. We will also add field constraints for the title and content. Add new file services/store/posts.go:

Package Store import "time" type Post struct {ID int Title string 'binding:" Required,min=3, Max =50"' Content string`binding:"required,min=5,max=5000"` CreatedAt time.Time ModifiedAt time.Time UserID int`json:"-"` }Copy the code

Users can have multiple blog posts, so we must add multiple relationships to the user structure. In services/store/users.go, edit the User structure:

type User struct {
  ID             int
  Username       string`binding:"required,min=5,max=30"`
  Password       string`pg:"-" binding:"required,min=7,max=32"`
  HashedPassword []byte`json:"-"`
  Salt           []byte`json:"-"`
  CreatedAt      time.Time
  ModifiedAt     time.Time
  Posts          []*Post `json:"-" pg:"fk:user_id,rel:has-many,on_delete:CASCADE"`
}
Copy the code

The ability to insert new post entries into the database will be implemented in services/store/posts.go:

func AddPost(user *User, post *Post) error { post.UserID = user.ID _, Err := db.model (post).returning (" * ").insert () if err! = nil {log.error ().err (Err).msg (" Error inserting new post ")} return Err}Copy the code

To create the post, we will add a new handler that will call the above function. Create a new file services/server/post.go:

Package Server import (net/ HTTP "pharos/services/ Store" "github.com/gin-gonic/gin") func createPost(CTX) *gin.Context) { post := new(store.Post) if err := ctx.Bind(post); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": err.Error()}) return } user, err := currentUser(ctx) if err ! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": err.Error()}) return } if err := store.AddPost(user, post); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": Err.error ()}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Post created successfully. ", "data" : Post,})}Copy the code

With the post creation handler ready, let’s add a new protected route for creating a post. In the services/server/router. Go, we will create a new group, the group will use in the previous section we implement the authorization of middleware. We’ll use the HTTP POST method to add a route /posts to the protected group:

func setRouter() *gin.Engine { // Creates default gin router with Logger and Recovery middleware already attached router Default() // Enables automatic redirection if the current route can't be matched but a // handler for the path with (without) the trailing slash exists. router.RedirectTrailingSlash = true // Create API route group api := Router.group ("/API ") {api.post (" /signup ", signup) api.post (" /signin ", SignIn)} Authorized := api.group ("/") authorized.Use(authorization) {authorized.POST(" /posts ", createPost) } router.NoRoute(func(ctx *gin.Context) { ctx.JSON(http.StatusNotFound, gin.H{}) }) return router }Copy the code

The recipe for all other CRUD (create, read, update, delete) methods is the same:

1. Implements the ability to communicate with the database to perform required operations
2. Implement the Gin handler, which uses the functions in Step 1
3. Add a route with a handler to the router

We’ve covered the creation part, so let’s move on to the next method, reading. We’ll implement a function to get all user posts from the services/ Store /posts.go database:

Func FetchUserPosts(user * user) error {err := db.model (user). Relation(" Posts ", func(q * orm.query) (* orm.query, Error) {return q.Order(" id ASC "), nil}). Select() if err! = nil {log.error ().err (Err).Msg(" Error reference ")} return Err}Copy the code

Add a file services/server/post.go:

func indexPosts(ctx *gin.Context) { user, err := currentUser(ctx) if err ! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": err.Error()}) return } if err := store.FetchUserPosts(user); err ! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": Err.error ()}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Posts touchsuccessfully. ", "data" : user.posts,})}Copy the code

To update posts, add these two functions to services/store/posts.go:

func FetchPost(id int) (*Post, error) { post := new(Post) post.ID = id err := db.Model(post).WherePK().Select() if err ! = nil {log.error ().err (Err).Msg(" Error post ") returnnil, Err} return post, fetching = nil {log.error ().err (Err). nil } func UpdatePost(post *Post) error { _, err := db.Model(post).WherePK().UpdateNotZero() if err ! = nil {log.error ().err (Err).msg (" Error updating post ")} return Err}Copy the code

Modify the services/server/post. Go

func updatePost(ctx *gin.Context) { jsonPost := new(store.Post) if err := ctx.Bind(jsonPost); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": err.Error()}) return } user, err := currentUser(ctx) if err ! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": err.Error()}) return } dbPost, err := store.FetchPost(jsonPost.ID) if err ! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": err... error ()}) return}. If the user ID! AbortWithStatusJSON(http.statusforbidden, gin.H{" error ": "Not Authorized."}) return} jsonPost.modifiedat = time.now () if err := Store.updatePost (jsonPost); err ! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": Err.error ()}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Post updated successfully. ", "data" : jsonPost,})}Copy the code

Delete services/store/posts.go

func DeletePost(post *Post) error { _, err := db.Model(post).WherePK().Delete() if err ! = nil {log.error ().err (Err).msg (" Error deleting post ")} return Err}Copy the code

In the services/server/post. Go

Func deletePost(CTX *gin.Context) {paramID := ctx.param (" id ") ID, err := strconv.atoi (paramID) if err! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": "Not Valid ID."}) return} user, err := currentUser(CTX) if err! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": err.Error()}) return } post, err := store.FetchPost(id) if err ! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": err... error ()}) return}. If the user ID! AbortWithStatusJSON(http.statusforbidden, gin.H{" error ": "Not authorized."}) return} if err := Store.deletePost (post); err ! = nil {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": Err.error ()}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Post deleted successfully. "})}Copy the code

One new thing you can notice here is paramID := ctx.param (“id”). We are using it to extract the ID parameter from the URL path.

Let’s add all these handlers to the router:

func setRouter() *gin.Engine { // Creates default gin router with Logger and Recovery middleware already attached router Default() // Enables automatic redirection if the current route can't be matched but a // handler for the path with (without) the trailing slash exists. router.RedirectTrailingSlash = true // Create API route group api := Router.group ("/API ") {api.post (" /signup ", signup) api.post (" /signin ", Authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") Authorized. POST(" /posts ", createPost) Authorized. PUT(" /posts ", updatePost) Authorized. DELETE(" /posts/:id ", deletePost) } router.NoRoute(func(ctx *gin.Context) { ctx.JSON(http.StatusNotFound, gin.H{}) }) return router }Copy the code

If the User has not yet posted, the user.posts field defaults to nil. This makes things complicated for the front end because it has to check for nil values, so it’s better to use empty slices. To do this, we’ll use AfterSelectHook, which executes after each Select() for User. This hook will be added to services/store/users.go:

var _ pg.AfterSelectHook = (*User)(nil)

func (user *User) AfterSelect(ctx context.Context) error {
  if user.Posts == nil {
    user.Posts = []*Post{}
  }
  return nil
}
Copy the code

X. Error and exception handling

If you try to create a new account with a Password that is too short, you will receive the Error Key: ‘user. Password’ Error:Field Validation for ‘Password’ failed on the ‘min’ tag. This is not really a good user experience, so it should be changed for a better user experience. Let’s see how we can turn this into our own custom error message. To this end, we will be in the services / / middleware server. Go file to create a new Gin handler function:

func customErrors(ctx *gin.Context) { ctx.Next() iflen(ctx.Errors) > 0 { for _, err := range ctx.Errors { // Check error type switch err.Type { case gin.ErrorTypePublic: // Show public errors only if nothing has been written yet if ! CTX. Writer. Written () {CTX. AbortWithStatusJSON (CTX. Writer. The Status (), gin. H {" error ": err.Error()}) } case gin.ErrorTypeBind: errMap := make(map[string]string) if errs, ok := err.Err.(validator.ValidationErrors); ok { for _, fieldErr := range []validator.FieldError(errs) { errMap[fieldErr.Field()] = customValidationError(fieldErr) } } status := http.StatusBadRequest // Preserve current status if ctx.Writer.Status() ! AbortWithStatusJSON(status, gin.H{" error ": errMap}) default: = http.statusok {status = ctx.writer.status ()} ctx.abortwithStatusjson (status, gin. Error().err (err.err).msg (" other Error ")}} // If there was no public or bind Error, display default 500 message if ! CTX. Writer. Written () {CTX. AbortWithStatusJSON (HTTP. StatusInternalServerError, gin. H {" error ": InternalServerError}) } } } func customValidationError(err validator.FieldError) string { switch err.Tag() { case "Required" : return FMt.sprintf (" %s is required. ", err.field ()) case "min" : Return fmt.Sprintf(" %s must be longer than or equal %s characters. ", err.field (), err.param ()) case "Max" : Return fmt.sprintf (" %s cannot be longer than %s characters. ", err.field (), err.param ()) Default: return err.error ()}}Copy the code

In the internal/server/server. Go InternalServerError defined constants

Const InternalServerError = "Something went wrong!"Copy the code

Let us in the services/server/router. Go in using the new Gin middleware:

func setRouter() *gin.Engine { // Creates default gin router with Logger and Recovery middleware already attached router Default() // Enables automatic redirection if the current route can't be matched but a // handler for the path with (without) the trailing slash exists. router.RedirectTrailingSlash = true // Create API route group api := Router.group ("/API ") api.use (customErrors) {api.post (" /signup ", gin.Bind(store.user {}), signup) api.post (" /signin ", Gin.bind (store.user {}), signIn)} Authorized := api.group ("/") authorized.Use(authorization) {authorized.GET(" /posts ", Authorized. POST(" /posts ", createPost) Authorized. PUT(" /posts ", updatePost) Authorized. DELETE(" /posts/:id ", deletePost) } router.NoRoute(func(ctx *gin.Context) { ctx.JSON(http.StatusNotFound, gin.H{}) }) return router }Copy the code

We are now using the customErrors middleware in the API group. But that’s not the only change. Note the update route for login and registration:

Api.post (" /signup ", gin.Bind(store.user {}), signup) api.post (" /signin ", gin.Bind(store.user {}), signin)Copy the code

With these changes, we will even try to bind the request data before clicking on signUp and signIn handlers, meaning that the handler will only be reached if the form is validated. With this setup, the handler does not need to consider binding errors because there are no binding errors if the handler is reached. With this in mind, let’s update these two handlers:

func signUp(ctx *gin.Context) { user := ctx.MustGet(gin.BindKey).(*store.User) if err := store.AddUser(user); err ! AbortWithStatusJSON(http.statusbadRequest, gin.H{" error ": Err.error ()}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Signed up successfully. ", "JWT" : generateJWT(user), }) } func signIn(ctx *gin.Context) { user := ctx.MustGet(gin.BindKey).(*store.User) user, err := store.Authenticate(user.Username, user.Password) if err ! = nil {ctx.AbortWithStatusJSON(http.statusunauthorized, gin.H{" error ": "}) return} ctx.JSON(http.statusok, gin.H{" MSG ":" Signed in successfully. ", "JWT" : generateJWT(user), }) }Copy the code

Our handlers are much simpler now, and they only handle database errors. If you try again to create an account with a username and password that is too short, you will see a more readable and descriptive error:

The above mentioned page login related error, if the database error we also gracefully handle, you try to use the existing username to create an account, ERROR #23505 Duplicate Key value revoking unique constraint “users_username_key” Unfortunately, there is no validator involved, and the PG module returns most errors as map[byte] strings, so this can be a bit tricky. One way is to manually check for each error condition by performing a database query. For example, to check if a user with a given user name already exists in the database, we can do this before attempting to create a new user:

Func AddUser(user * user) error {err = db.model (user).Where(" username =? , user.Username).Select() if err ! = nil {return errors.New(" Username already exists. ")}... }Copy the code

The problem is that it becomes very boring. You need to do this for every error case in every function that communicates with the database. Most importantly, we added database queries unnecessarily. In this simple example, there will now be 2 database queries for each successful user creation, instead of 1. Another option is to try a query and parse if an error occurs. This is the tricky part, because we need to process each error type using regular expressions to extract the relevant data needed to create a more user-friendly custom error message. So let’s get started. As mentioned earlier, pg errors are primarily map[byte] strings, so when you try to create a user account with an existing user name, for this particular error you get a Map object in the following figure:

B5566213-AA61-4F8F-8116-C7CC3210C971.png

To extract the relevant data, we will use fields 82 and 110. The error type will be read from field 82, and we will extract the column name from field 110. Let’s add these functions to services/store/store.go:

func dbError(_err interface{}) error {
  if _err == nil {
    returnnil
  }
  switch _err.(type) {
  case pg.Error:
    err := _err.(pg.Error)
    switch err.Field(82) {
    case “_bt_check_unique”:
      return errors.New(extractColumnName(err.Field(110)) + “ already exists.”)
    }
  case error:
    err := _err.(error)
    switch err.Error() {
    case “pg: no rows in result set”:
      return errors.New(“Not found.”)
    }
    return err
  }
  return errors.New(fmt.Sprint(_err))
}

func extractColumnName(text string) string {
  reg := regexp.MustCompile(`.+_(.+)_.+`)
  if reg.MatchString(text) {
    return strings.Title(reg.FindStringSubmatch(text)[1])
  }
  return “Unknown”
}
Copy the code

With this, we can call the dbError() function from services/store/users.go:

Func AddUser(user * user) error {... _, err = db.model (user).returning (" * ").insert () if err! Error().err (Err).msg (" Error inserting new user ") return dbError(Err)} returnnil}Copy the code

If we had used an existing user name, we could have prompted a more elegant prompt. The other thing that needs to be done gracefully is to turn off the service. We want to modify the services/server/server. Go

Package Server import (" Context "" Errors" "NET/HTTP" "OS" "OS/Signal" "Pharos /services/conf" "Pharos/Services /database" "Pharos/services/store" and "the syscall" "time" "github.com/rs/zerolog/log") const InternalServerError = "Something went Wrong!" func Start(cfg conf.Config) { jwtSetup(cfg) store.SetDBConnection(database.NewDBOptions(cfg)) router := setRouter() Server := &http. server {Addr: cfg.Host + ":" + cfg.Port, Handler: router, } // Initializing the server in a goroutine so that // it won't block the graceful shutdown handling below gofunc() {if  err := server.ListenAndServe(); err ! = nil && errors.Is(err, Http.errserverclosed) {log.error ().err (Err).msg (" Server ListenAndServe Error ")}}() // Wait for interrupt signal to gracefully shutdown the server with // a timeout of 5 seconds. quit := make(chan os.Signal) // kill (no param) default Send syscall.SIGTERM // kill -2 is syscall.SIGINT // kill -9 is syscall.SIGKILL but can't be caught, So don't need to add it signal.notify (quit, syscall.sigint, syscall.sigterm) <-quit log.info ().msg (" Shutting down Server... ) // The context is used to inform the server it has 5 seconds to finish // the request it is currently handling ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel() if err := server.Shutdown(ctx); err ! = nil {log.fatal ().err (Err).msg (" Server forced to shutdown ")} log.info ().msg (" Server no longer intended. ")}Copy the code

Xi. Test

Writing unit and integration tests is an important part of software development, and some basic application issues need to be ensured before you start writing tests related, for example, the main thing to do is to create test databases. This will be done using the development database schema that has been created.

We’ll start by creating a new test configuration. Add the following function to services/conf/conf.go:

func NewTestConfig() Config {
  testConfig := NewConfig()
  testConfig.DbName = testConfig.DbName + "_test"
  return testConfig
}
Copy the code

This creates the same new configuration as the usual one, but with the _test suffix appended to the database name. Refer to the previous example of adding a database to add a new database. The test database will be named pharOS_test.

DROP DATABASE IF EXISTS pharos_test;
CREATE DATABASE pharos_test WITH TEMPLATE pharos;
Copy the code

Create a database called pharOS_test, which is required every time you change the open database.

Each test case must be independent of the other use cases, so I should well for each test case using the new database, so will be at the beginning of each test case creation and invoke the top-up database functions, in this function we will reset all the table names, clean up all the tables, reset the counter, to ensure that all ID sequence starting from 1, We can add functions to services/store/store.go

func ResetTestDatabase() { // Connect to test database SetDBConnection(database.NewDBOptions(conf.NewTestConfig())) // Empty all tables and restart sequence counters tables := []string{" users ", "posts"} for _, table := range tables {_, Err := db.exec (FMT.Sprintf(" DELETE FROM %s; , table)) if err ! = nil {log.panic ().err (Err).str (" table ", table).msg (" Error clearing test database ")} _, Err = db.exec (FMT.Sprintf(" ALTER SEQUENCE %s_id_seq RESTART; , table)) } }Copy the code

One thing we need to do in most tests is set up the test environment and create new users. We don’t want to repeat this in every test case. Let’s create a file services/store/main_test.go and add an auxiliary function:

Package Store import (" pharos/services/conf "" pharos/services/store" "github.com/gin-gonic/gin") func testSetup() *gin.Engine {gin.SetMode(gin.TestMode) store.resetTestDatabase () CFG := conf.newConfig (" dev ") jwtSetup(CFG) return SetRouter (CFG)} func addTestUser() (*User, error) {User := &User{Username: "batman", Password: Secret123,} err := AddUser(user) return user, err}Copy the code

Now that the preparations are complete, we can start adding tests. Let’s create a new file services/store/users_test.go and create the first test:

Package store import (" testing "and" github.com/stretchr/testify/assert ") func TestAddUser (t * testing in t) {the z3c.testsetup () user, err := addTestUser() assert.NoError(t, err) assert.Equal(t, 1, user.ID) assert.NotEmpty(t, user.Salt) assert.NotEmpty(t, user.HashedPassword) }Copy the code

Another test we can add for user accounts is when a user tries to create an account using an existing username:

func TestAddUserWithExistingUsername(t *testing.T) {
  testSetup()
  user, err := addTestUser()
  assert.NoError(t, err)
  assert.Equal(t, 1, user.ID)

  user, err = addTestUser()
  assert.Error(t, err)
  assert.Equal(t, “Username already exists.”, err.Error())
}
Copy the code

To test the Authenticate() function, we will create three tests: successful authentication, authentication with an invalid username, and authentication with an invalid password:

func TestAuthenticateUser(t *testing.T) { testSetup() user, err := addTestUser() assert.NoError(t, err) authUser, err := Authenticate(user.Username, user.Password) assert.NoError(t, err) assert.Equal(t, user.ID, authUser.ID) assert.Equal(t, user.Username, authUser.Username) assert.Equal(t, user.Salt, authUser.Salt) assert.Equal(t, user.HashedPassword, authUser.HashedPassword) assert.Empty(t, authUser.Password) } func TestAuthenticateUserInvalidUsername(t *testing.T) { testSetup() user, Err := addTestUser() assert.noerror (t, err) authUser, err := Authenticate(" invalid ", user.password) assert.error (t, err) assert.Nil(t, authUser) } func TestAuthenticateUserInvalidPassword(t *testing.T) { testSetup() user, Err := addTestUser() assert.noerror (t, err) authUser, err := Authenticate(user.username, "invalid") assert.error (t, err) assert.Nil(t, authUser) }Copy the code

Finally, we’ll test the FetchUser() function with two tests: successfully fetching and retrieving non-existent users:

func TestFetchUser(t *testing.T) { testSetup() user, err := addTestUser() assert.NoError(t, err) fetchedUser, err := FetchUser(user.ID) assert.NoError(t, err) assert.Equal(t, user.ID, fetchedUser.ID) assert.Equal(t, user.Username, fetchedUser.Username) assert.Empty(t, fetchedUser.Password) assert.Equal(t, user.Salt, fetchedUser.Salt) assert.Equal(t, user.HashedPassword, fetchedUser.HashedPassword) } func TestFetchNotExistingUser(t *testing.T) { testSetup() fetchedUser, Nil(t, fetchedUser) assert.Equal(t, "Not found.", err.error ())} err := FetchUser(1) assert.Error(t, err) assert.Nil(t, fetchedUser) assert.Equal(t, "Not found.", err.error ())}Copy the code

The above test function only tests database communication, but our router and handler are not tested here. To do this, we will need another set of tests. First, we should create more helper functions. We will create a new file services/server/main_test.go:

package server import ( "bytes" "encoding/json" "net/http" "net/http/httptest" "pharos/services/store" "strings" "github.com/gin-gonic/gin" "github.com/rs/zerolog/log" ) func testSetup() *gin.Engine { gin.SetMode(gin.TestMode) store.ResetTestDatabase() jwtSetup() return setRouter() } func userJSON(user store.User) string { body, Err := json.Marshal(map[string]interface{}{" Username ": user.username," Password ": user.password,}) if err! Err(Err).msg (" Error marshalling JSON body. ")} returnString (body)} func jsonRes(body * bytes.buffer)  map[string]interface{} { jsonValue := &map[string]interface{}{} err := json.Unmarshal(body.Bytes(), jsonValue) if err ! Err(Err).Msg(" Error unmarshalling JSON body. ")} return *jsonValue} func performRequest(router) *gin.Engine, method, path, body string) *httptest.ResponseRecorder { req, err := http.NewRequest(method, path, strings.NewReader(body)) if err ! Nil {log.panic ().err (Err).msg (" Error creating new Request ")} rec := httptest.newRecorder () Req.header. Add(" content-type ", "application/json") router.ServeHTTP(rec, req) return rec}Copy the code

With the exception of the last one, performRequest(), most of these functions are nothing new. In this function, we use the HTTP package to create a new request and the HttpTest package to create a new logger. We also need to add a Content-Type header with a value of Application/JSON to our test request. We are now ready to process the test request using the passed router and log the response using the logger. Now let’s see how these functions are actually used. Create a new file services/server/user_test.go:

Package server import (" net/HTTP "" pharos/services/store" "testing" "github.com/stretchr/testify/assert") func TestSignUp(t *testing.T) {router := testSetup() Body := userJSON(store.user {Username: "batman", Password: "Secret123",}) rec := performRequest(Router, "POST", "/ API /signup", body) assert.equal (t, http.statusok, Rec.code) assert.equal (t, "Signed up successfully.", jsonRes(rec.body)[" MSG "]) assert.notempty (t, JsonRes (rec) Body) [] "JWT")}Copy the code

It is important to note that the test cases are run sequentially without parallelism. If run at the same time, they can affect each other because the database is empty for each test case. If your machine has multiple kernels, Go defaults to using multiple Goroutines to run tests. To ensure that only one goroutine is used, add the -p 1 option. This means that you should run the test using the following command:

Go test -p 1./internal/...Copy the code

Xii. Deployment

Our server is complete and almost ready to deploy, this will be done using Docker. Notice that I said it’s almost ready, so let’s see what’s missing. All along, we’ve used the React development server, which listens on port 8181 and redirects all requests to our back-end port 8080. This makes a lot of development sense because it makes it easier to develop both the front end and the back end at the same time, as well as debug front-end reaction applications. But we don’t need it in production, it makes more sense to just run our back-end servers and provide static front-end files to clients. Therefore, instead of using the NPM start command to start the React development server, we will use the command NPM run build in the app/ directory to build optimized front-end files for production. This will create a new directory app/build/ and all the files needed for production. Now we must also indicate to our back end where to find these files so that we can serve them. Use(static.serve (“/”, static.localfile (“./app/build”, true)))). Of course, we want to do this only when starting the server in a PROD environment, so we need to update the files a little bit.

First, we’ll update the Parse() function in services/cli/cli.go to return the environment value as a string:

Func Parse() string {flag.usage = Usage env := flag.string (" env ", "dev", `Sets run environment. Possible values are "dev" and "prod"`) flag.Parse() logging.ConfigureLogger(*env) if *env == "Prod" {logging.setginlogtofile ()} return *env}Copy the code

Then we’ll update the Config struct NewConfig() function to be able to receive and set the environment values:

pe Config struct {
  Host       string
  Port       string
  DbHost     string
  DbPort     string
  DbName     string
  DbUser     string
  DbPassword string
  JwtSecret  string
  Env        string
}

func NewConfig(env string) Config {
  …
  return Config{
    Host:       host,
    Port:       port,
    DbHost:     dbHost,
    DbPort:     dbPort,
    DbName:     dbName,
    DbUser:     dbUser,
    DbPassword: dbPassword,
    JwtSecret:  jwtSecret,
    Env:        env,
  }
}
Copy the code

Now we can update services/cli/main.go to receive the env value from the CLI and send it to the new configuration creation that will be used to start the server:

func main() {
  env := cli.Parse()
  server.Start(conf.NewConfig(env))
}
Copy the code

The next thing we need to do is update the router to be able to receive configuration parameters and set it to serve static files on startup in production mode:

Package Server import (" NET/HTTP "pharos/services/conf" pharos/services/store "github.com/gin-contrib/static "github.com/gin-gonic/gin") func setRouter(CFG conf.config) *gin.Engine {// Creates default gin router with Logger and  Recovery middleware already attached router := gin.Default() // Enables automatic redirection if the current route Can "t be matched but a / / handler for the path with (without) the trailing slash the exists. The router. RedirectTrailingSlash = True // Serve static files to frontend if server is started in production environment if CFG.Env == "prod" { The router. Use (static. Serve ("/", the static LocalFile (". / app/build ", True)))} // Create API route group API := router.group ("/API ") api.Use(customErrors) {api.post (" /signup ", Gin. Bind (store. User {}), signUp) API. POST ("/signin ", gin. Bind (store) User {}), Authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") authorization.get (" /posts ") IndexPosts) Authorized.POST(" /posts ", gin.Bind(store.post {}), createPost) Authorized.PUT(" /posts ", Gin. Bind (store. Post {}), updatePost) authorized. DELETE ("/posts / : id ", deletePost) } router.NoRoute(func(ctx *gin.Context) { ctx.JSON(http.StatusNotFound, gin.H{}) }) return router }Copy the code

The last line to update is in the migrations/main.go file

store.SetDBConnection(database.NewDBOptions(conf.NewConfig()))
Copy the code

Instead of

Store. SetDBConnection (database. NewDBOptions (conf. NewConfig (" dev ")))Copy the code

It’s not done yet. You must also update all tests that use configuration and router Settings.

Everything is now ready for Docker deployment. Docker is outside the scope of this guide, so I won’t go into the details of Dockerfile,.dockerignore, and docker-comemess.yml content.

First we will create the.dockerignore file in the project root directory:

# This file
.dockerignore

# Git files
.git/
.gitignore

# VS Code config dir
.vscode/

# Docker configuration files
docker/

# Assets dependencies and built files
app/build/
app/node_modules/

# Log files
logs/

# Built binary
cmd/pharos/pharos

# ENV file
.env

# Readme file
README.md
Copy the code

Now create a new directory docker/ with two files Dockerfile and docker-comemage. yml. The contents of the Dockerfile will be:

FROM node:16 AS frontendBuilder # set app work dir WORKDIR /pharos # copy assets files to the container COPY app/ . # set app/ as work dir to build frontend static files WORKDIR /pharos/app RUN npm install RUN npm run build FROM Golang :1.16.3 AS backendBuilder # set app work dir WORKDIR /go/ SRC /pharos # copy all files to the container copy.. # build app executable RUN CGO_ENABLED=0 GOOS=linux go build -o cmd/pharos/pharos cmd/pharos/main.go # build migrations Executable RUN CGO_ENABLED=0 GOOS= Linux go build-o migrations/*. Go FROM alpine:3.14 # Create a group and user deploy RUN addgroup -S deploy && adduser -S deploy -G deploy ARG ROOT_DIR=/home/deploy/pharos WORKDIR ${ROOT_DIR} RUN chown deploy:deploy ${ROOT_DIR} # copy static assets file from frontend build copy - from=frontendBuilder - chown=deploy:deploy /pharos/build./app/build # Copy app and migrations executables from Backend Builder copy - the from = backendBuilder - chown = deploy: deploy/go/SRC/pharos/migrations/migrations. / migrations/COPY - from = backendBuilder - chown = deploy: deploy/go/SRC/pharos/CMD/pharos/pharos. # set user deploy as current user user deploy # start app CMD [ ". / pharos ", "- env", "prod"]Copy the code

Docker-comemage. yml contains the following contents:

Version: "3" services: pharos: image: Kramat /pharos env_file: -.. /.env environment: PHAROS_DB_HOST: db depends_on: - db ports: - ${PHAROS_PORT}:${PHAROS_PORT} db: image: postgres environment: POSTGRES_USER: ${PHAROS_DB_USER} POSTGRES_PASSWORD: ${PHAROS_DB_PASSWORD} POSTGRES_DB: ${PHAROS_DB_NAME} ports: - ${PHAROS_DB_PORT}:${PHAROS_DB_PORT} volumes: - postgresql:/var/lib/postgresql/pharos - postgresql_data:/var/lib/postgresql/pharos/data volumes: postgresql: {} postgresql_data: {}Copy the code

With all the files required for Docker deployment now in place, let’s see how to build a Docker image and deploy it. First, we’ll pull the Postgres image from the official Docker container repository:

docker pull postgres
Copy the code

The next step is to build the PharOS image. Run in the project root (change the DOCKER_ID using your own docker ID) :

docker build -t DOCKER_ID/pharos -f docker/Dockerfile .
Copy the code

To create pharos and DB containers with resources, run:

cd docker/
docker-compose up -d
Copy the code

This will start both containers, and you can check their status by running Docker PS. Finally, we need to run the migration. Open the shell by running it in the Pharos container:

Docker-compose run -rm pharos shCopy the code

Inside the container, we can run the migration as before:

cd migrations/
./migrations init
./migrations up
Copy the code

We’ve done it. You can open localhost:8181 in your browser to check that everything is ok, which means you should be able to create accounts and add new posts:

There are a lot of things that need to be done to improve a website, but these are just some of the tips.