Automatic GC times out

[ghost]

Version information: 0.4.5-dev-e4be1b2

Type: Bug

Priority: P2

Description:

The automatic GC times out. StorageMax is 54G, the repo is 60G big, so the timeout for GC should be something like 6 minutes.

From jupiter:

23:41:29.807 WARNI   corerepo: pre-GC: Maximum storage limit exceeded. Maybe unpin some files? gc.go:187
23:41:29.807  INFO   corerepo: Watermark exceeded. Starting repo GC... gc.go:191
23:42:58.626 ERROR   corerepo: context deadline exceeded gc.go:165

Note how the context times out after after less than 1.5m already.

[ghost]

The context that's timing out is in GC.maybeGC():

		log.Info("Watermark exceeded. Starting repo GC...")

		// 1 minute is sufficient for ~1GB unlink() blocks each of 100kb in SSD
		_ctx, cancel := context.WithTimeout(ctx, time.Duration(gc.SlackGB)*time.Minute)
		defer cancel()

		if err := GarbageCollect(gc.Node, _ctx); err != nil {
			return err
		}

StorageMax is 60G, and StorageGC is 54G, so SlackGB is 6G, and thus the timeout hits after 6 minutes. The time between the "starting gc" log line and the context timeout was only about 1.5 minutes when I witnessed this live.

@whyrusleeping @kevina @Kubuxu what would you say about just upping the timeout? HDDs and Flash will be much much slower than what's anticipated here, and apparantly even DigitalOcean's shared SSDs are too slow.

[ghost]

Or maybe just make it configurable.

[whyrusleeping]

That metric just seems so wrong? why does the slack space get to dictate how long this has to run?

Why does this even need a timeout? that seems odd...

[ghost]

Okay I'll look into removing the timeout, or making it something absurdly high