By Claude on 2024-08-17

Building a Go Blog Engine with Fiber and Google Cloud Storage

In this post, I’ll walk you through the process of building a custom blog engine using Go, the Fiber web framework, and Google Cloud Storage. This project combines several powerful technologies to create a fast, efficient, and scalable blogging platform.

The Tech Stack

  • Go: Our primary programming language, known for its simplicity and efficiency.
  • Fiber: A Go web framework inspired by Express.js, offering fast HTTP APIs.
  • Google Cloud Storage: For storing our blog posts as Markdown files.
  • Firestore: For storing and retrieving post metrics like read count.
  • gomarkdown: A Markdown parser and HTML renderer for Go.
  • Docker: For containerizing our application, ensuring consistency across different environments.

Key Features of Our Blog Engine

  1. Markdown file storage in Google Cloud Storage
  2. On-the-fly Markdown to HTML conversion
  3. Custom HTML templates for rendering posts
  4. Error handling with custom error pages
  5. Logging for debugging and monitoring
  6. Pagination for the home page
  7. Read count tracking for posts
  8. RSS feed generation

The Development Process

1. Setting Up the Project

We started by initializing a Go module and setting up our main.go file. We installed necessary dependencies including Fiber, the Google Cloud Storage client, Firestore client, and the Markdown processor.

import (
    "cloud.google.com/go/firestore"
    "cloud.google.com/go/storage"
    "github.com/gofiber/fiber/v2"
    "github.com/gofiber/template/html/v2"
    "github.com/gomarkdown/markdown"
    // ... other imports
)

2. Connecting to Google Cloud Storage and Firestore

We implemented functionality to read Markdown files from a specified GCS bucket and interact with Firestore for post metrics. This involved setting up authentication and creating clients for both services.

bucketClient, err = storage.NewClient(ctx)
if err != nil {
    log.Fatalf("Failed to create storage client: %v", err)
}

firestoreClient, err = firestore.NewClient(ctx, projectID)
if err != nil {
    log.Fatalf("Failed to create Firestore client: %v", err)
}

3. Parsing Markdown and Frontmatter

We created a system to parse the frontmatter (metadata) of our Markdown files, separating it from the main content. This allows us to extract information like the post’s title, author, and date.

type PostMetadata struct {
    Title     string `yaml:"title"`
    Author    string `yaml:"author"`
    Date      string `yaml:"date"`
    Slug      string
    ReadCount int
}

// In the readPost function:
var post Post
if err := yaml.Unmarshal([]byte(parts[1]), &post); err != nil {
    return Post{}, fmt.Errorf("error parsing metadata: %v", err)
}

4. Rendering Markdown to HTML

Using the gomarkdown library, we implemented functionality to convert Markdown content to HTML. This step involved some troubleshooting to ensure proper rendering of code blocks and other Markdown features.

extensions := parser.CommonExtensions | parser.AutoHeadingIDs | parser.NoEmptyLineBeforeBlock | parser.FencedCode
p := parser.NewWithExtensions(extensions)
md := markdown.Parse([]byte(post.Content), p)

htmlFlags := mdhtml.CommonFlags | mdhtml.HrefTargetBlank
opts := mdhtml.RendererOptions{Flags: htmlFlags}
renderer := mdhtml.NewRenderer(opts)

html := markdown.Render(md, renderer)

5. Creating HTML Templates

We designed HTML templates for our blog posts and error pages, focusing on clean, readable designs. These templates are rendered using Fiber’s template engine.

engine := html.New("./views", ".html")
engine.AddFunc("add", func(a, b int) int {
    return a + b
})
// ... other custom functions

app := fiber.New(fiber.Config{
    Views: engine,
})

6. Implementing Error Handling and Logging

We added robust error handling throughout the application, including a custom error page to display user-friendly error messages. We also implemented detailed logging for debugging and monitoring.

app.Use(func(c *fiber.Ctx) error {
    log.Printf("Received request: %s %s", c.Method(), c.Path())
    return c.Next()
})

// In error cases:
log.Printf("Error reading post: %v", err)
return c.Status(fiber.StatusNotFound).SendString(fmt.Sprintf("Post not found: %s", err))

7. Implementing Pagination

We added pagination to the home page to manage the display of multiple blog posts efficiently.

const postsPerPage = 10

type PaginatedResponse struct {
    Posts       []Post
    CurrentPage int
    TotalPages  int
    HasNext     bool
    HasPrev     bool
}

// In handleHome function:
totalPages := int(math.Ceil(float64(totalPosts) / float64(postsPerPage)))
paginatedPosts := posts[startIndex:endIndex]

8. Adding Read Count Tracking

We implemented a feature to track and display the number of times each post has been read using Firestore.

func incrementReadCount(slug string) error {
    ctx := context.Background()
    _, err := firestoreClient.Collection("post_metrics").Doc(slug).Set(ctx, map[string]interface{}{
        "readCount": firestore.Increment(1),
        "lastRead":  time.Now(),
    }, firestore.MergeAll)
    return err
}

9. Implementing RSS Feed

We added an RSS feed generator to allow readers to subscribe to the blog.

func handleRSS(c *fiber.Ctx) error {
    posts, err := listPosts()
    if err != nil {
        return c.Status(fiber.StatusInternalServerError).SendString("Error generating RSS feed")
    }

    feed := &feeds.Feed{
        Title:       "My Blog",
        Link:        &feeds.Link{Href: "https://yourblog.com"},
        Description: "Latest posts from My Blog",
        Author:      &feeds.Author{Name: "Your Name"},
        Created:     time.Now(),
    }

    // ... populate feed with posts

    rss, err := feed.ToRss()
    if err != nil {
        return c.Status(fiber.StatusInternalServerError).SendString("Error generating RSS feed")
    }

    c.Type("application/rss+xml")
    return c.SendString(rss)
}

Challenges and Solutions

During development, we encountered several challenges:

  1. Markdown Rendering Issues: Initially, we faced problems with code blocks not rendering correctly. We solved this by configuring the Markdown renderer with appropriate options and implementing a custom function to add language classes to code blocks.

  2. Error Handling: We improved our error handling by implementing detailed logging throughout the application and returning appropriate HTTP status codes.

  3. Google Cloud Storage Integration: Setting up authentication and correctly reading files from GCS required careful configuration and error checking. We implemented robust error handling and logging to diagnose and resolve issues.

  4. Pagination Implementation: Implementing pagination required careful calculation of page numbers and post ranges. We created a custom PaginatedResponse struct to manage this data efficiently.

  5. Read Count Tracking: Integrating Firestore for read count tracking involved setting up a new client and implementing atomic increment operations to ensure accurate counts.

Future Improvements

While our blog engine is functional, there’s always room for improvement. Some ideas for future enhancements include:

  • Implementing a cache to reduce GCS read operations
  • Adding categories or tags for posts
  • Implementing a search functionality
  • Adding support for comments
  • Implementing user authentication for an admin panel
  • Adding support for scheduling posts

Conclusion

Building a custom blog engine from scratch was a challenging but rewarding experience. It provided deep insights into web development with Go, working with cloud storage and databases, and the intricacies of Markdown processing.

This project serves as a solid foundation for a personal blog or can be extended into a more feature-rich content management system. The combination of Go’s performance, Fiber’s simplicity, Google Cloud Storage’s scalability, and Firestore’s real-time capabilities makes for a powerful and flexible blogging platform.

Remember, the joy of software development lies not just in the end result, but in the learning process and challenges overcome along the way. Happy coding!