How to read write from to a file using go

In Go, reading and writing files are primarily handled through the io and os packages in the standard library. The following outlines basic file operation steps and example code.

How to Write Files

To write to a file in Go, utilize the Create and OpenFile functions from the os package to create or open a file, and employ the Write or WriteString methods to write data. If the file does not exist, Create will create it. OpenFile allows specifying different flags to determine the mode (e.g., read-only, write-only, or append) and permissions.

go
package main

import (
	log
	os
)

func main() {
	// Create or open file
	file, err := os.Create("example.txt")
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()

	// Write byte data
	byteSlice := []byte("Hello Golang!\n")
	_, err = file.Write(byteSlice)
	if err != nil {
		log.Fatal(err)
	}

	// Write string data
	_, err = file.WriteString("Writing string data to a file.\n")
	if err != nil {
		log.Fatal(err)
	}
}

How to Read Files

When reading files, use the os.Open function to open the file and then read its contents using the io package or the bufio package. The Scanner type provided by the bufio package is commonly used for reading text files separated by newline characters.

go
package main

import (
	bufio
	fmt
	log
	os
)

func main() {
	// Open file
	file, err := os.Open("example.txt")
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()

	// Use bufio.NewReader to read file contents
	reader := bufio.NewReader(file)
	for {
		// Read a line
		line, err := reader.ReadString('\n')
		if err != nil {
			// If reached end of file, break loop
			if err.Error() == "EOF" {
				break
			}
			log.Fatal(err)
		}
		fmt.Print(line)
	}
}

Error Handling

In the above examples, you may notice that error checking is performed after each file operation. This is because reading and writing files can encounter various errors, such as the file not existing or insufficient permissions. In Go, error handling is crucial; always check each operation that might fail.

File Closing

After completing file operations, use the defer statement to ensure the file is properly closed. The defer statement executes when the function containing it ends, ensuring the file is closed even if an error occurs.

This covers the basic methods for reading and writing files in Go. In practical applications, more complex file handling may be involved, such as reading large files in chunks or using concurrency to speed up file processing.

New Method

Starting from Go 1.16, use os.ReadFile to load the file into memory and os.WriteFile to write from memory to the file. Note that ioutil.ReadFile is deprecated and now calls os.ReadFile.

Be cautious with os.ReadFile as it loads the entire file into memory.

go
package main

import "os"
import "log"

func main() {
    b, err := os.ReadFile("input.txt")
    if err != nil {
        log.Fatal(err)
    }

    // `b` contains the entire content of your file.
    // This writes it to Standard Output.
    os.Stdout.Write(b)

    // You can also write it to a file as a whole.
    err = os.WriteFile("destination.txt", b, 0644)
    if err != nil {
        log.Fatal(err)
    }
}

Here is a list compatible with Go 1, detailing all methods for reading and writing files in Go.

Due to recent changes in the file API, most other answers are not compatible with Go 1. To be honest, bufio also overlooked an important point.

Starting from the Basics

go
package main

import (
    "io"
    "os"
)

func main() {
    // open input file
    fi, err := os.Open("input.txt")
    if err != nil {
        panic(err)
    }
    // close fi on exit and check for its returned error
    defer func() {
        if err := fi.Close(); err != nil {
            panic(err)
        }
    }()

    // open output file
    fo, err := os.Create("output.txt")
    if err != nil {
        panic(err)
    }
    // close fo on exit and check for its returned error
    defer func() {
        if err := fo.Close(); err != nil {
            panic(err)
        }
    }()

    // make a buffer to keep chunks that are read
    buf := make([]byte, 1024)
    for {
        // read a chunk
        n, err := fi.Read(buf)
        if err != nil && err != io.EOF {
            panic(err)
        }
        if n == 0 {
            break
        }

        // write a chunk
        if _, err := fo.Write(buf[:n]); err != nil {
            panic(err)
        }
    }
}

Here, I use os.Open and os.Create, which are convenient wrappers for os.OpenFile. We typically do not need to call OpenFile directly.

Note the handling of EOF. Read attempts to fill buf on each call, returning io.EOF when it reaches the end of the file. In this case, buf still retains the data. Subsequent calls to Read return zero bytes read along with io.EOF. Any other error causes a panic.

Using bufio

go
package main

import (
    "bufio"
    "io"
    "os"
)

func main() {
    // open input file
    fi, err := os.Open("input.txt")
    if err != nil {
        panic(err)
    }
    // close fi on exit and check for its returned error
    defer func() {
        if err := fi.Close(); err != nil {
            panic(err)
        }
    }()
    // make a read buffer
    r := bufio.NewReader(fi)

    // open output file
    fo, err := os.Create("output.txt")
    if err != nil {
        panic(err)
    }
    // close fo on exit and check for its returned error
    defer func() {
        if err := fo.Close(); err != nil {
            panic(err)
        }
    }()
    // make a write buffer
    w := bufio.NewWriter(fo)

    // make a buffer to keep chunks that are read
    buf := make([]byte, 1024)
    for {
        // read a chunk
        n, err := r.Read(buf)
        if err != nil && err != io.EOF {
            panic(err)
        }
        if n == 0 {
            break
        }

        // write a chunk
        if _, err := w.Write(buf[:n]); err != nil {
            panic(err)
        }
    }

    if err = w.Flush(); err != nil {
        panic(err)
    }
}

Here, bufio serves as a buffer since we have minimal interaction with the data. In most other scenarios—especially for text files—it provides a well-designed API for easy and flexible reading and writing, while internally handling buffering, which is highly useful.

Note: The following code is compatible with older Go versions (Go 1.15 and earlier). Things have changed (since Go 1.16, ioutil has been deprecated). For the new approach, see this answer.

Using ioutil

go
package main

import (
    "io/ioutil"
)

func main() {
    // read the whole file at once
    b, err := ioutil.ReadFile("input.txt")
    if err != nil {
        panic(err)
    }

    // write the whole body at once
    err = ioutil.WriteFile("output.txt", b, 0644)
    if err != nil {
        panic(err)
    }
}

Trivial! But only use it when you're certain you're not dealing with large files.