Wednesday, Dec 27, 2023

golang

This marks the second entry of a series of posts where I delve into Golang idioms and best practices that I've come across. The exploration will encompass insights into practices that, in hindsight, I wish I had adopted earlier in my journey with Golang.

First part can be found here, "Golang Idioms and Best Practices: Part 1".

Project Structure Guidelines

In the early stages of a new Go project, prioritizing its structure may not be essential. However, as the project evolves, maintaining a cleaner codebase becomes increasingly important. Otherwise, there is a risk of encountering hidden dependencies and global state issues if project cleanup and structure are deferred.

While there isn't an officially designated "standard" for Go project layouts, the Go community has established certain practices. The Go standard project structure repository can be accessed here, serving as a reference based on community-driven conventions.

Depending on the specific needs of the application or library, I recommend considering these guidelines as just that "guidelines".

I won't cover every Go project structure idiom from github.com/golang-standards/project-layout, but three that I consider crucial are: cmd, pkg, and internal.

cmd/*

The cmd/* directory is designated for placing one or more entry points for individual applications, each with a corresponding executable name (e.g., /cmd/foo or /cmd/bar). It is crucial not to include any exported code in this directory that might be imported by other packages. In such cases, prefer using the pkg/ and internal/ directories instead.

internal/*

The /internal directory serves as a designated space for housing local unexported components meant exclusively for use within the project to which it belongs. Since Go version 1.5, there's a built-in enforcement mechanism preventing the import of "internal" packages from outside the subtree where the package is defined.

For instance, consider two distinct projects: github.com/mjyocca/go-library and github.com/mjyocca/go-application. In this scenario, the go-application project would be restricted from importing go-library/internal/a, while still being permitted to import go-library/pkg/b.

pkg/*

The /pkg directory is designated for common components that can be utilized within the same project or external Go applications. This is in opposition to the internal/ directory, which confines usage to within the project and doesn't allow external access.

While there is no inherent functionality associated with this directory name, it serves as an indication to developers that the code within this subtree is intended for use by consuming applications.

Auxillary Directories

In my view, anything beyond the cmd, pkg, and internal directories is subject to individual or team discretion and is entirely contingent on the specific use case. For a more detailed list, comprehensive documentation is available at the Go Standard project layout repository.

Some of these directories include:

• api/: OpenAPI, Swagger, JSON schema files, protocol definition files.
• build/: Packaging and continuous integration. Examples: Docker, Cloud AMI, CI configurations and scripts.
• configs/: Configuration file templates or default configs. Example: consult-templates
• docs/: Design, user documents, diagrams, godoc generated documentation.
• examples/: Examples on how to use the project's application or public libraries.

Go Workspaces

Requires Go v1.18, you can learn more about Go Workspaces here, and get hands-on with this tutorial.

This isn't technically an idiom; instead, it's a useful feature. It can improve productivity when managing multiple Go projects simultaneously or when working on an upstream library while concurrently developing an application that relies on it.

Alternatives to Go Workspcaces would be to either release a new module version or modify the go.mod file using the replace directive to reference a local file system version of the unpublished module (e.g. go mod edit -replace github.com/library/package=Users/{user}/Projects/package).

Go Workspaces are controlled by a new file, go.work, in the root of the workspace directory. You can create this file by running go work init, along with optional space-separated arguments. Modules can be added later by using go work use [moddir] or by directly editing the go.work file.

NewStruct() constructor

In Go, the default zero value for certain primitives may not be suitable, necessitating the use of an initializing constructor. This approach also serves as a pattern to prevent redundant logic.

The idiomatic practice here is to prefix the returning type with newStruct(...) *Struct or NewStruct(...) *Struct for exported methods.

type Driver struct {
	Name string // zero value: ""
	Age int // zero value: 0
}
 
// or `NewDriver` if exported 
func newDriver(name string) *Driver {
	d := Driver{
		Name: name
		Age: 16 // sensible default
	}
	return &d
}

Accept Interfaces and Return Structs

This practice and recommended approach facilitate the implementation of a widely-used design pattern known as Dependency Injection, improving code maintainability and delineating external logic from the implementation.

Enabling this pattern involves decoupling components by empowering the consumer to define the desired methods. Meanwhile, the producer package accepts the interface as an argument and yields a concrete struct type.

Choosing an interface as an argument, rather than a concrete type, augments flexibility in the implementation. This is because the producing package can accommodate any type adhering to the interface, fostering greater code reuse and simplifying refactors.

Returning a concrete type, such as struct{}, ensures that future maintainers precisely understand the type they are working with. If the producer were to return a non-concrete type, like an interface, the consumer would only have access to the methods defined in the interface, excluding any additional methods specified in the concrete type.

Logger Example

For example, this producer package accepts an interface as an argument for New that the consumer can define as a concrete type. The producer can then accept the concrete type as long as it fulfills the Log method.

package producer
 
// interface
type Logger interface {
	Log(message string)
}
 
type Config struct {
	logger Logger
}
 
func (c *Config) Add(x, y int) int {
	c.logger.Log("adding x: %d and y: %d", x, y)
	return x + y;
}
 
func New(logger Logger) *Config {
	return &Config{logger}
}

package main
 
import (
	"fmt"
	"test/producer"
)
 
type MyLogger struct{}
 
// satisfies Log(message string) interface method
func (m *MyLogger) Log(message string) {
	fmt.Println(message)
}
 
// but can contain other fields and methods as well
func (m *MyLogger) Error(message string) {
	fmt.Printf("error: %s\n", message)
}
 
func main() {
	logger := &MyLogger{}
	lib := producer.New(logger)
	lib.Add(1, 1)
}

The MyLogger{} concrete type can have additional fields and methods outside of the required Log(message string) method.

Database/Storer Example

NewUserService fn accepts argument s of type UserStorer interface, and returns a pointer to a UserService struct

package db
 
import "database/sql"
 
type Store struct {
	db *sql.DB
}
 
func NewDB() *Store {
	return &Store{
		db: &sql.DB{},
	}
}
 
func (s *Store) Insert(item interface{}) error {
 
	//... to-do
	return nil
}
 
func (s *Store) Get(id string, val interface{}) error {
 
	//... to-do
	return nil
}

package service
 
import "context"
 
type UserStorer interface {
	Insert(item interface{}) error
	Get(id string, val interface{}) error
}
 
type User struct {
	ID    string
	Email string
}
 
type UserService struct {
	store UserStorer
}
 
func NewUserService(s UserStorer) *UserService {
	return &UserService{
		store: s,
	}
}
 
func (u *UserService) CreateUser(ctx context.Context, user User) error {
	err := u.store.Insert(user)
	if err != nil {
		return err
	}
	return nil
}
 
func (u *UserService) RetrieveUser(ctx context.Context, id string) (interface{}, error) {
	var user User
	err := u.store.Get(id, &user)
	if err != nil {
		return nil, err
	}
	return user, nil
}

package main
 
import (
	"context"
	"fmt"
	"test/db"
	"test/service"
)
 
func main() {
	ctx := context.Background()
	// store injected into user service
	store := db.NewDB()
	// user service struct, can now use it's exposed methods
	useService := service.NewUserService(store)
 
	user := &service.User{}
	if err := useService.CreateUser(ctx, user); err != nil {
		fmt.Println(fmt.Errorf("error creating user: %s", err))
	}
 
	fmt.Println(fmt.Printf("User created: %+v", user))
}

Exceptions to the rule

When to accept structs

If you require access to fields or methods unique to a particular concrete type, it is advisable to accept that specific type. For instance, if you need to access a field exclusive to MyStruct, consider accepting a *MyStruct type.

When to return interfaces

If there is a need to return a type that satisfies multiple interfaces, it is recommended to return an interface. For instance, if a type implements both the Writer and Logger interfaces, consider returning a Writer or Logger interface.

Functional Options Pattern

When creating packages and libraries in Go, the necessity of providing sufficient configuration options can be cumbersome for both the author and the developer consuming the package.

In the absence of design patterns (e.g. Functional Options Pattern), two approaches can be adopted. One is defining a new constructor for each configuration option, and the other is accepting a Configuration struct as an argument. Both methods enable the customization of configuration but come with certain pitfalls.

New Constructor for each configuration Example

In the case of defining a new constructor for each configuration option, is the least flexible option but can lead to less frequently breaking changes. Usually the library or package starts with a generic New(...) constructor defined with the required arguments. As more options are added, and so are the new distinct constructors. Otherwise breaking changes are introduced by modifying the signature of existing constructors.

package server
 
import "time"
 
type Logger interface {
	Log(message string)
}
 
type defaultLogger struct{}
 
func (l *defaultLogger) Log(message string) {}
 
type Server struct {
	hostname string
	port     int
	timeout  time.Duration
	logger   Logger
}
 
func New(hostname string, port int) *Server {
	return &Server{hostname, port, time.Minute, &defaultLogger{}}
}
 
func NewWithTimeout(hostname string, port int, timeout time.Duration) *Server {
	return &Server{hostname, port, timeout, &defaultLogger{}}
}
 
func NewWithTimeoutAndLogger(hostname string, port int, timeout time.Duration, logger Logger) *Server {
	return &Server{hostname, port, timeout, logger}
}

Configuration Config Example

By utilizing the Config{} struct option, the necessity of appending new constructor methods for each added option is circumvented. This approach also sidesteps the challenge of introducing breaking changes to the constructor signature when new options are added or removed. On the contrary, when there's a requirement to modify the structure of the options Config{}, it can result in breaking changes when old fields are removed and potential breaking changes when new struct fields are added.

Additionally, to implement the aforementioned solution, we also had to make each field exported (capitalized) to enable the consuming package to set properties on the struct Config{} directly. This, in turn, permits the consuming package to directly manipulate its properties.

It's worth mentioning that if the consumer opts for implicit assignment of the struct fields, the addition of new fields would introduce a breaking change.

Example of implicit assignment for struct fields:

s := server.New(server.Config{"localhost", 1234, 30 * time.Second, logger})

Example

package server
 
import (
	"time"
)
 
type Logger interface {
	Log(message string)
}
 
type defaultLogger struct{}
 
func (l *defaultLogger) Log(message string) {}
 
type Server struct {
	config Config
}
 
func (s *Server) Start() error {
	time.Sleep(time.Minute)
	return nil
}
 
type Config struct {
	Hostname string
	Port     int
	Timeout  time.Duration
	Logger   Logger
}
 
func New(config Config) *Server {
	return &Server{config}
}

Functional Options Pattern Example

The Functional Options Pattern offers a flexible API that benefits both authors and developers, addressing the downsides associated with previous solutions. Leveraging Go's variadic functions, which permit a function to be invoked with any number of trailing arguments denoted by ellipsis ..., enables the creation of an extendable and flexible solution without introducing breaking changes.

type Options func(*Server)
 
func New(options ...Options) *Server {
	s := &Server{
		// set defaults
	}
	// iterate through variadic functions
	// each invocation receives pointer to the shared Server{} struct
	for _, opts := range options {
		opts(s)
	}
	return s
}
 
func WithHostname(hostname string) func(*Server) {
	return func(s *Server) {
		s.hostname = hostname
	}
}
 
func WithLogger(logger Logger) func(*Server) {
	return func(s *Server) {
		s.logger = logger
	}
}
 
func WithPort(port int) func(*Server) {
	return func(s *Server) {
		s.port = port
	}
}

main.go

package main
 
import (
	"fmt"
	"test/server"
)
 
func main() {
	logger := &MyLogger{}
	server.New(
		server.WithHostname("localhost"),
		server.WithPort(1234),
		server.WithLogger(logger),
	)
}

Through the combination of a variadic function and the type Options func(*Server) type signature, we can pass any number of options in any order into the New constructor. This approach offers several advantages, including keeping fields unexported, abstracting configuration, avoiding breaking changes, and fostering a readable and self-documenting API design for developers.

Alternatively, there are other patterns that can be employed with similar advantages such as the Builder Pattern.

Context Values Best Practices

Limit storage in Context

From the context package docs, "Use context Values only for request-scoped data that transits processes and APIs, not for passing optional parameters to functions."

As per the package documentation, don't use context.WithValue(...) as a generalized data store for optional values, but rather for request-scoped data that spans the lifecycle of a process such as an API request.

Types of data to store in Context

Limit storing data with context.WithValue(...) to one of the following cases:

• Session ID's
• Tracing / Spans
• Telemetry
• Logging
• Authentication / Security Credentials

Additional Context Values Best Practices

Avoid primitive keys for storing Values in Context

The WithValue method in the context package accepts three parameters: the parent context, a key and a value with an any type func WithValue(parent Context, key, val any) Context. The Value(key) method context method receiver also accepts any type for the value key.

Don't use primitives such as a string or other built-in types as the key when storing data in context WithValue(...). Doing so will prevent collisions between packages using context. Rather than relying on primitives like strings, opt for a concrete type, such as a user-defined struct or type alias.

package main
 
import (
	"context"
	"fmt"
)
 
// using typed-alias to prevent collisions
type favContextKey string
 
func main() {
	ctx := context.Background()
	ctx.WithValue(ctx, favContextKey("foo"), "bar")
	val := ctx.Value(favContextKey("foo"))
	fmt.Println(val) // output: "bar"
}
 

Add compile time checks

When utilizing context.WithValue and context.Value(), you forfeit compile-time checks due to the method signatures permitting the use of the any type.

To reintroduce compile-time checks, you can implement more verbose type assertions and create user-defined typed methods for inserting and extracting values from context.

package trace
 
import "context"
 
type contextKeyT string
 
var contextKey = contextKeyT("my_trace_id")
 
type Trace struct{}
 
func New() *Trace {
	return &Trace{}
}
 
// inserts typed value into context
func NewContext(ctx context.Context, tr *Trace) context.Context {
	return context.WithValue(ctx, contextKey, tr)
}
 
// extracts typed value from context
func FromContext(ctx context.Context) (tr Trace, ok bool) {
	// extracts value along with asserting its type.
	tr, ok = ctx.Value(contextKey).(Trace)
	return
}

NewContext: accepts a typed value of trace{} to insert in context and returns a new context. FromContext: extracts typed value from context and asserts its type.

Example consuming the trace package

func processing(ctx context.Context) {
	tracer, valid := trace.FromContext(ctx)
	if !valid {
		fmt.Println("tracer not found")
	}
	fmt.Println("tracer found from context", tracer)
}
 
func main() {
	tr := trace.New()
	ctx := trace.NewContext(context.Background(), tr)
	processing(ctx)
}

Avoid adding Context as a struct field member

As a general rule, avoid adding a Context to a struct type as field member, but instead pass a ctx paramter to each method receiver for that type that depends on it. I've personally have done this for some edge cases such as when methods need access to context but the method signature must match an interface for a standard or third party library.

type MyStruct struct {
	ctx context.Context
}
// avoid
func (m *MyStruct) Create(message string) {
	ctx := m.ctx
}
// instead
func (m *MyStruct) Create(ctx context.Context, message string) {}

Conclusion

We've touched on some of the idiomatic best practices when writing Go, in regards to:

• Project structure in regards to cmd, pkg, and internal
• Go Workspaces
• NewStruct constructor
• Accept Interfaces and Return Structs
• Functional Options Pattern
• Context Values Best Practices

Related Articles

• "Golang Idioms and Best Practices: Part 1".

Code on :)