Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optimized ReadAll function #229

Merged
merged 10 commits into from
Oct 2, 2024
Merged

Commits on Oct 1, 2024

  1. Add optimized ReadAll function

    This adds an optimized io.ReadAll that uses a bytes.Buffer + io.Copy
    internally, improving the buffer growth ratio over the runtime append
    and taking advange of io.WriterTo when available.
    
    The only scenario where this optimizes version is slower than io.ReadAll is
    when reading < 512 bytes of a reader that does not implement io.WriterTo.
    
    For now I only updated tests to use the new function as updating the code
    requires more careful consideration.
    
    While at it, migrate from the deprecated ioutil.ReadAll to io.ReadAll.
    
    goos: linux
    goarch: amd64
    pkg: github.com/elastic/elastic-agent-libs/iobuf
    cpu: Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz
    BenchmarkReadAll/size_32B/io.ReadAll/WriterTo-32         	 7479658	       162.4 ns/op	     512 B/op	       1 allocs/op
    BenchmarkReadAll/size_32B/io.ReadAll/Reader-32           	 7000012	       169.3 ns/op	     512 B/op	       1 allocs/op
    BenchmarkReadAll/size_32B/ReadAll/WriterTo-32            	10630838	       112.3 ns/op	     112 B/op	       2 allocs/op
    BenchmarkReadAll/size_32B/ReadAll/Reader-32              	 2111246	       567.3 ns/op	    1584 B/op	       3 allocs/op
    BenchmarkReadAll/size_64B/io.ReadAll/WriterTo-32         	 7154652	       163.2 ns/op	     512 B/op	       1 allocs/op
    BenchmarkReadAll/size_64B/io.ReadAll/Reader-32           	 7223264	       166.1 ns/op	     512 B/op	       1 allocs/op
    BenchmarkReadAll/size_64B/ReadAll/WriterTo-32            	10400562	       113.5 ns/op	     112 B/op	       2 allocs/op
    BenchmarkReadAll/size_64B/ReadAll/Reader-32              	 2129949	       558.7 ns/op	    1584 B/op	       3 allocs/op
    BenchmarkReadAll/size_512B/io.ReadAll/WriterTo-32        	 2843871	       419.1 ns/op	    1408 B/op	       2 allocs/op
    BenchmarkReadAll/size_512B/io.ReadAll/Reader-32          	 2871580	       413.1 ns/op	    1408 B/op	       2 allocs/op
    BenchmarkReadAll/size_512B/ReadAll/WriterTo-32           	 4976233	       241.5 ns/op	     560 B/op	       2 allocs/op
    BenchmarkReadAll/size_512B/ReadAll/Reader-32             	 2183186	       552.9 ns/op	    1584 B/op	       3 allocs/op
    BenchmarkReadAll/size_10KB/io.ReadAll/WriterTo-32        	  142633	      8235 ns/op	   46080 B/op	      10 allocs/op
    BenchmarkReadAll/size_10KB/io.ReadAll/Reader-32          	  148326	      8229 ns/op	   46080 B/op	      10 allocs/op
    BenchmarkReadAll/size_10KB/ReadAll/WriterTo-32           	  574903	      2210 ns/op	   10288 B/op	       2 allocs/op
    BenchmarkReadAll/size_10KB/ReadAll/Reader-32             	  147210	      7995 ns/op	   32304 B/op	       7 allocs/op
    BenchmarkReadAll/size_100KB/io.ReadAll/WriterTo-32       	   13171	     90853 ns/op	  514304 B/op	      18 allocs/op
    BenchmarkReadAll/size_100KB/io.ReadAll/Reader-32         	   12892	     91787 ns/op	  514304 B/op	      18 allocs/op
    BenchmarkReadAll/size_100KB/ReadAll/WriterTo-32          	   51472	     22420 ns/op	  106544 B/op	       2 allocs/op
    BenchmarkReadAll/size_100KB/ReadAll/Reader-32            	   21568	     55070 ns/op	  261680 B/op	      10 allocs/op
    BenchmarkReadAll/size_1MB/io.ReadAll/WriterTo-32         	    1220	    983276 ns/op	 5241098 B/op	      27 allocs/op
    BenchmarkReadAll/size_1MB/io.ReadAll/Reader-32           	    1089	    990818 ns/op	 5241100 B/op	      27 allocs/op
    BenchmarkReadAll/size_1MB/ReadAll/WriterTo-32            	    4153	    294507 ns/op	 1048627 B/op	       2 allocs/op
    BenchmarkReadAll/size_1MB/ReadAll/Reader-32              	    1195	    944781 ns/op	 4193852 B/op	      14 allocs/op
    mauri870 committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    9fde46e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c5876b2 View commit details
    Browse the repository at this point in the history
  3. update ReadAll doc

    mauri870 committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    85f0f87 View commit details
    Browse the repository at this point in the history
  4. fix linter error in doc

    mauri870 committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    9595f21 View commit details
    Browse the repository at this point in the history
  5. nolint for http.Serve

    mauri870 committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    57a455b View commit details
    Browse the repository at this point in the history
  6. fix license headers

    mauri870 committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    3d2be93 View commit details
    Browse the repository at this point in the history
  7. fix code duplication

    mauri870 committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    f310b03 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    4d5d1da View commit details
    Browse the repository at this point in the history
  9. remove os.CreateTemp

    mauri870 committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    11ab9fd View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. Configuration menu
    Copy the full SHA
    7f4272a View commit details
    Browse the repository at this point in the history