Flat structure and layered architecture. Build your Go application

Building a great application starts with its structure. The structure of an application sets the tone for the development of the application, so it is important to get it right from the start.

Go is a relatively simple language that makes no comments about the structure of the application. In this article, we’ll explore two main ways you can structure your Go application.

Before we continue, it’s important to note that no structure is perfect for all applications. Some of the things we cover may not be appropriate for your library or project. However, you should know what’s available so you can easily decide how best to build your application.

Build the Go application with a flat structure

This approach to building a project places all files and packages in the same directory.

At first, this might seem like a bad way to structure a project, but there are ways to build it that work perfectly. An example project using a flat structure would have the following structure.

flat_app/
  main.go
  lib.go
  lib_test.go
  go.mod
  go.sum

Copy the code

The main advantage of using this structure is that it is easy to operate. All created packages are in the same directory, so they can be easily modified and used as needed.

This structure is best used for building libraries, simple scripts, or simple CLI applications. HttpRouter, a widely used routing library for building apis, uses a similar flat structure.

However, a major drawback is that as the project becomes more complex, it becomes almost impossible to maintain. For example, this structure is not suitable for building REST apis because apis have different components that make them function well, such as controllers, models, configurations, and middleware. These components should not all be kept in one file directory.

Ideally, you should use a flat structure when launching an application. Once you’re uncomfortable with clutter, you can upgrade to any other project structure.

Build a simple API with a flat structure

To demonstrate the flat structure, let’s build an API for a Notepad application.

Create a new directory for this project by running.

mkdir notes_api_flat

Copy the code

The directory is named notes_API_flat because the application may have changes using other structures, which we’ll cover later.

Now initialize the project.

go mod init github.com/username/notes_api_flat

Copy the code

This application will allow users to store notes. We will use SQLite3 for note storage and Gin for routing. Run the following snippet to install them.

go get github.com/mattn/go-sqlite3
go get github.com/gin-gonic/gin

Copy the code

Next, create the following files.

main.go: The entry point of the application
models.go: Manages database access
migration.go: Manages table creation

After they are created, the folder structure should look like this.

notes_api_flat/
  go.mod
  go.sum
  go.main.go
  migration.go
  models.go

Copy the code

write`migration.go`

Add the following to migration.go to create the table that will store our notes.

package main
import (
  "database/sql"
  "log"
)
const notes = `
  CREATE TABLE IF NOT EXISTS notes (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    title VARCHAR(64) NOT NULL,
    body MEDIUMTEXT NOT NULL,
    created_at TIMESTAMP NOT NULL,
    updated_at TIMESTAMP NOT NULL
  )
`
func migrate(dbDriver *sql.DB) {
  statement, err := dbDriver.Prepare(notes)
  if err == nil {
    _, creationError := statement.Exec()
    if creationError == nil {
      log.Println("Table created successfully")
    } else {
      log.Println(creationError.Error())
    }
  } else {
    log.Println(err.Error())
  }
}

Copy the code

In the above snippet, we declare that the package is main. Note that we can’t set it to something different from main.go because they are in the same directory. Therefore, everything in each file is globally available because all files are in the same package.

Notice that we imported the packages needed to interact with the SQL, as well as a log package to log any errors that occurred.

Next, we have an SQL query that creates a note table with the following fields: ID,title,body, CREATED_AT, and updated_AT.

Finally, we defined the function Migrate, which executes the query written above and prints any errors that occur during the process.

create`models.go`

Add the following to Models.go.

package main import ( "log" "time" ) type Note struct { Id int `json:"id"` Title string `json:"title"` Body string `json:"body"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` } func (note *Note) create(data NoteParams) (*Note, error) { var created_at = time.Now().UTC() var updated_at = time.Now().UTC() statement, _ := DB.Prepare("INSERT INTO notes (title, body, created_at, updated_at) VALUES (? ,? ,? ,?) ") result, err := statement.Exec(data.Title, data.Body, created_at, updated_at) if err == nil { id, _ := result.LastInsertId() note.Id = int(id) note.Title = data.Title note.Body = data.Body note.CreatedAt = created_at note.UpdatedAt = updated_at return note, err } log.Println("Unable to create note", err.Error()) return note, err } func (note *Note) getAll() ([]Note, error) { rows, err := DB.Query("SELECT * FROM notes") allNotes := []Note{} if err == nil { for rows.Next() { var currentNote Note rows.Scan( &currentNote.Id, &currentNote.Title, &currentNote.Body, &currentNote.CreatedAt, &currentNote.UpdatedAt) allNotes = append(allNotes, currentNote) } return allNotes, err } return allNotes, err } func (note *Note) Fetch(id string) (*Note, error) { err := DB.QueryRow( "SELECT id, title, body, created_at, updated_at FROM notes WHERE id=?" , id).Scan( &note.Id, &note.Title, &note.Body, &note.CreatedAt, &note.UpdatedAt) return note, err }Copy the code

The Model contains a definition of the note structure and three methods that allow notes to interact with the database. The annotation structure contains all the data that an annotation can have and should be synchronized with columns in the database.

The create method is responsible for creating a new note and returns the newly created note and any errors that occur during this process.

The getAll method takes all the notes in the database as a fragment and returns it along with any errors that occurred in the process.

The Fetch method retrieves a specific note from its ID. All of these methods can be used to retrieve notes directly in the future.

Complete the API in Go

The last remaining piece of the API is routing. Modify main.go to include the following code.

package main import ( "database/sql" "log" "net/http" "github.com/gin-gonic/gin" _ "github.com/mattn/go-sqlite3" ) // Create this to store instance to SQL var DB *sql.DB func main() { var err error DB, err = sql.Open("sqlite3", "./notesapi.db") if err ! = nil { log.Println("Driver creation failed", err.Error()) } else { // Create all the tables migrate(DB) router := gin.Default() router.GET("/notes", getAllNotes) router.POST("/notes", createNewNote) router.GET("/notes/:note_id", getSingleNote) router.Run(":8000") } } type NoteParams struct { Title string `json:"title"` Body string `json:"body"` } func createNewNote(c *gin.Context) { var params NoteParams var note Note err := c.BindJSON(&params) if err == nil { _, creationError := note.create(params) if creationError == nil { c.JSON(http.StatusCreated, gin.H{ "message": "Note created successfully", "note": note, }) } else { c.String(http.StatusInternalServerError, creationError.Error()) } } else { c.String(http.StatusInternalServerError, err.Error()) } } func getAllNotes(c *gin.Context) { var note Note notes, err := note.getAll() if err == nil { c.JSON(http.StatusOK, gin.H{ "message": "All Notes", "notes": notes, }) } else { c.String(http.StatusInternalServerError, err.Error()) } } func getSingleNote(c *gin.Context) { var note Note id := c.Param("note_id") _, err := note.Fetch(id) if err == nil { c.JSON(http.StatusOK, gin.H{ "message": "Single Note", "note": note, }) } else { c.String(http.StatusInternalServerError, err.Error()) } }Copy the code

Here, we import all the packages we need. Note the final import.

"github.com/mattn/go-sqlite3"

Copy the code

This code snippet is required to work with SQLite, although it is not used directly. The main function first initializes the database and exits if it fails. The database instance is stored on DB global variables so that it can be easily accessed.

Next, we migrate the tables by calling the Migrate function, which is defined in migrations.go.

We don’t need to import anything to use this function because it is available globally in the main package.

Next, define the route. We only need three routes.

aGETRequest to the/notesRetrieves all notes that have been created and stored in the database.
aPOSTRequest to the/notes, create a new note and persist it to the database.
aGETRequest to the/note/:note_idTo retrieve a note.id

These routes have separate handlers that use the note model to perform the required database operations.

Advantages of using a flat structure

We can see that by using a flat structure, we can quickly build simple apis without having to manage multiple packages. This is especially useful to library authors, since most modules need to be part of the base package.

Disadvantages of using a flat structure

While there are many benefits to using a flat structure, it is not the best choice when building aN API. First, this structure has considerable limitations in that it automatically makes functions and variables globally available.

There is no real separation of concerns. We tried to separate the model from migration and routing, but it was almost impossible because they could still be accessed directly from each other. This can cause one file to modify items it shouldn’t, or change without the knowledge of another file, so the application is not easy to maintain.

The structure we will introduce next solves many of the problems of using flat structures.

Use layered architecture (classic MVC architecture) in Go.

This structure groups files according to their functionality. The packages (models) that handle communication with the database are grouped and stored differently from the packages that handle requests from the route.

Let’s look at what a hierarchical architecture looks like.

layered_app/
  app/
    models/
      User.go         
    controllers/
      UserController.go
  config/
    app.go
  views/
    index.html
  public/
    images/
      logo.png
  main.go
  go.mod
  go.sum

Copy the code

Notice the separation. Because it’s easier to maintain such structured projects, and you’ll have less clutter in your code using the MVC structure.

While the layered structure is not ideal for building simple libraries, it is good for building apis and other large applications. This is usually the default structure for applications built using Revel, a popular Go framework for building REST APIs.

Update the Go application with a layered architecture

Now that you have seen an example project using a layered architecture, let’s upgrade our project from a flat structure to an MVC structure.

Create a new folder named notes_API_LAYERED and initialize a Go module in it by running the following code snippet.

mkdir notes_api_layered
go mod init github.com/username/notes_api_layered

Copy the code

Install the required SQLite and Gin packages.

go get github.com/mattn/go-sqlite3
go get github.com/gin-gonic/gin

Copy the code

Now, update the folder structure of your project to look like this.

notes_api_layered/
  config/
    db.go
  controllers/
    note.go
  migrations/
    main.go
    note.go
  models/
    note.go
  go.mod
  go.sum
  main.go

Copy the code

As you can see from the new folder structure, all files are arranged according to their functionality. All models are in the model directory, as are migration, controller, and configuration.

Next, we refactor the work we did in the flat structure implementation into this new structure.

Starting with the database configuration, add the following in config/db.go.

package config
import (
  "database/sql"
  _ "github.com/mattn/go-sqlite3"
)
var DB *sql.DB
func InitializeDB() (*sql.DB, error) {
  // Initialize connection to the database
  var err error
  DB, err = sql.Open("sqlite3", "./notesapi.db")
  return DB, err
}

Copy the code

Here, we declare a package called config and import all the associated libraries to communicate with the database. Note that we can declare multiple packages because they are not all in the same directory.

Next, we create a DB variable that will hold the connection to the database, since each model has a different database instance, which is not ideal. Note: Variable or function names that start with a capital letter mean they should be exported.

We then declare and export an InitializeDB function that opens the database and stores the database reference in a DB variable.

Once we have set up the database, we can move on to the migration. We have two files in the Migrations folder: main.go and note.go.

Main.go is responsible for loading the queries to be executed and then executing them, while note.go contains SQL queries specifically for Notes tables.

If we have other tables, for example, a table for annotations, they will also have a migration file that contains the query to create the annotation table.

Now add the following in migrations/note.go.

package migrations
const Notes = `
CREATE TABLE IF NOT EXISTS notes (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  title VARCHAR(64) NOT NULL,
  body MEDIUMTEXT NOT NULL,
  created_at TIMESTAMP NOT NULL,
  updated_at TIMESTAMP NOT NULL
)
`

Copy the code

Update migrations/main.go to include.

package migrations
import (
  "database/sql"
  "log"
  "github.com/username/notes_api_layered/config"
)
func Run() {
  // Migrate notes
  migrate(config.DB, Notes)
  // Other migrations can be added here.
}
func migrate(dbDriver *sql.DB, query string) {
  statement, err := dbDriver.Prepare(query)
  if err == nil {
    _, creationError := statement.Exec()
    if creationError == nil {
      log.Println("Table created successfully")
    } else {
      log.Println(creationError.Error())
    }
  } else {
    log.Println(err.Error())
  }
}

Copy the code

As explained earlier, migrations/main.go handles loading the query from the individual migration files and running it when the Run method is called. Migrate is a private function and cannot be used outside of this module. The only function that is output to the outside world is Run.

After running the migration, we need to update the model. The changes between the hierarchical and flat implementations here are fairly minor.

All methods for external use should be exported, and all references to DB should be changed to config.db.

After applying these changes, models/note.go should look like this.

package models
import (
  "log"
  "time"
  "github.com/username/notes_api_layered/config"
)
type Note struct {
  Id        int       `json:"id"`
  Title     string    `json:"title"`
  Body      string    `json:"body"`
  CreatedAt time.Time `json:"created_at"`
  UpdatedAt time.Time `json:"updated_at"`
}
type NoteParams struct {
  Title string
  Body  string
}
func (note *Note) Create(data NoteParams) (*Note, error) {
  var created_at = time.Now().UTC()
  var updated_at = time.Now().UTC()
  statement, _ := config.DB.Prepare("INSERT INTO notes (title, body, created_at, updated_at) VALUES (?, ?, ?, ?)")
  result, err := statement.Exec(data.Title, data.Body, created_at, updated_at)
  if err == nil {
    id, _ := result.LastInsertId()
    note.Id = int(id)
    note.Title = data.Title
    note.Body = data.Body
    note.CreatedAt = created_at
    note.UpdatedAt = updated_at
    return note, err
  }
  log.Println("Unable to create note", err.Error())
  return note, err
}
func (note *Note) GetAll() ([]Note, error) {
  rows, err := config.DB.Query("SELECT * FROM notes")
  allNotes := []Note{}
  if err == nil {
    for rows.Next() {
      var currentNote Note
      rows.Scan(
        &currentNote.Id,
        &currentNote.Title,
        &currentNote.Body,
        &currentNote.CreatedAt,
        &currentNote.UpdatedAt)
      allNotes = append(allNotes, currentNote)
    }
    return allNotes, err
  }
  return allNotes, err
}
func (note *Note) Fetch(id string) (*Note, error) {
  err := config.DB.QueryRow(
    "SELECT id, title, body, created_at, updated_at FROM notes WHERE id=?", id).Scan(
    &note.Id, &note.Title, &note.Body, &note.CreatedAt, &note.UpdatedAt)
  return note, err
}

Copy the code

We have already declared a new package, models, and imported from github.com/username/notes_api_layered/config configuration. With this, we can access DB, which will be initialized once the InitializeDB function is called.

Changes to the controller are also fairly minor, mainly involving exporting functions and importing models from newly created models.

Change the code snippet.

var note Note
var params NoteParams

Copy the code

To this.

var note models.Note
var params models.NoteParams

Copy the code

With this modification, the controller will look something like this.

package controllers import ( "net/http" "github.com/gin-gonic/gin" "github.com/username/notes_api_layered/models" ) type  NoteController struct{} func (_ *NoteController) CreateNewNote(c *gin.Context) { var params models.NoteParams var note models.Note err := c.BindJSON(&params) if err == nil { _, creationError := note.Create(params) if creationError == nil { c.JSON(http.StatusCreated, gin.H{ "message": "Note created successfully", "note": note, }) } else { c.String(http.StatusInternalServerError, creationError.Error()) } } else { c.String(http.StatusInternalServerError, err.Error()) } } func (_ *NoteController) GetAllNotes(c *gin.Context) { var note models.Note notes, err := note.GetAll() if err == nil { c.JSON(http.StatusOK, gin.H{ "message": "All Notes", "notes": notes, }) } else { c.String(http.StatusInternalServerError, err.Error()) } } func (_ *NoteController) GetSingleNote(c *gin.Context) { var note models.Note id := c.Param("note_id") _, err := note.Fetch(id) if err == nil { c.JSON(http.StatusOK, gin.H{ "message": "Single Note", "note": note, }) } else { c.String(http.StatusInternalServerError, err.Error()) } }Copy the code

From the code snippet above, we convert functions to methods so that they pass through notecontroller.create instead of controller.create. This is to allow for the possibility of having multiple controllers, which would be the case for most modern applications.

Finally, we update main.go to reflect the results of the refactoring.

package main import ( "log" "github.com/gin-gonic/gin" "github.com/username/notes_api_layered/config" "github.com/username/notes_api_layered/controllers" "github.com/username/notes_api_layered/migrations" ) func main() { _, err := config.InitializeDB() if err ! = nil { log.Println("Driver creation failed", err.Error()) } else { // Run all migrations migrations.Run() router := gin.Default() var noteController controllers.NoteController router.GET("/notes", noteController.GetAllNotes) router.POST("/notes", noteController.CreateNewNote) router.GET("/notes/:note_id", noteController.GetSingleNote) router.Run(":8000") } }Copy the code

After refactoring, we have a compact main package that imports the required packages: Config,controllers, and Models. It then initializes the database by calling config.initializedb ().

Now we can turn to routing. The route should be updated to handle requests with the newly created note controller.

The benefits of using a hierarchical structure in Go

The biggest benefit of using a hierarchical structure is that you can understand what each file and/or folder is doing from the directory structure. There is also an obvious separation of concerns, as each package has a single function to perform.

With a layered structure, the project can be easily extended. For example, if you add a new feature that allows users to comment on notes, this will be easy to implement because all the groundwork has already been done. In this case, just create the Model, migration, controller, and update the route. This feature has been added.

Disadvantages of using a hierarchical structure

This structure can be overkill for simple projects, requiring a lot of planning before implementation.

conclusion

In summary, we’ve seen that choosing a structure for your Go application depends on the project you’re building, how complex the project is, and how long you plan to work on it.

For creating simple projects, a flat structure is fine. However, when the project is more complex, you must step back and rethink and choose a more appropriate structure for your application.

Other popular constructs for building Go applications are domain-driven development and hexagonal architecture. If your application continues to scale, it may be worth learning this as well.

The postFlat Structure vs. Layered Architecture: Building your Go Application first appeared on The LogRocket blog.

Flat structure and layered architecture. Build your Go application

Build the Go application with a flat structure

Build a simple API with a flat structure

writemigration.go

createmodels.go