Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go #12808

Closed
jefferai opened this issue Oct 1, 2015 · 15 comments
Closed
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@jefferai
Copy link

jefferai commented Oct 1, 2015

I've been trying to figure out how to make my programs build statically by default without specifying -a -installsuffix cgo -ldflags '-extldflags "-static" -s for every project. I've read through #9344, and supposedly the change to make dynamic builds the default is only for released versions of Go. Accordingly, I tried the following:

  1. Install Go onto my system from a package manager
  2. Check out go1.5.1 from Git
  3. Set GOROOT to point to the checked-out version of Go, and GOROOT_BOOTSTRAP to the system version of Go
  4. cd src; ./all.bash -- successful
  5. rm -rf $GOPATH/pkg
  6. Start a new shell with GOPATH, GOROOT, and PATH all set appropriately. I can verify that I'm using the self-compiled version of Go (which itself was built dynamically), but binaries produced with my self-built Go are still being built dynamically without passing in -a -installsuffix cgo -ldflags '-extldflags "-static" -s

Is this a bug, or did I misinterpret the requirements in #9344?

Thanks!

@ianlancetaylor
Copy link
Contributor

I can't tell if this is a bug report or not. We use the issue tracker for actual bugs and proposed changes. Please ask questions on the golang-nuts mailing list or on https://forum.golangbridge.org/. Thanks.

I can't see what you are referring to in #9344. In any case, when you check out go1.5.1, you are getting a released version of Go. To get a non-released version of Go, you need to check out something that is not a release. But of course those are not supported.

If you build your own version of Go with CGO_ENABLED=0 set in the environment, you should get a version of Go that builds statically by default.

@jefferai
Copy link
Author

jefferai commented Oct 1, 2015

@ianlancetaylor I couldn't tell if it was a bug either, because I wasn't sure if the workflow that was described was expected to work, or not. After reading what you said, I do think it's at least a documentation issue.

#9344 suggests that the reason for dynamic linking in released versions is because they are installed to system locations that normal users don't have access to change. This is confusing, because this should be a toggle flipped by packagers when building binary packages for release, but instead is (apparently) something enforced in the code in Git. IOW, it's not clear that "released version" means "tagged in Git" as opposed to "downloadable release from the web site".

@jefferai
Copy link
Author

jefferai commented Oct 1, 2015

@ianlancetaylor I built Go from go1.5.1 tag with CGO_ENABLED=0 set (which appears to be the default anyways), and after setting that as GOROOT it still builds dynamic programs by default, unless I also set CGO_ENABLED=0 before building the program.

@ianlancetaylor
Copy link
Contributor

I'm still not sure what you're referring to in #9344, but it doesn't much matter. What matters is the documentation.

Never set GOROOT. If it is in your environment, take it out. If you build a release, run bin/go from that release.

CGO_ENABLED=0 is not the default for a native build.

Setting CGO_ENABLED when building the Go sources is documented in src/make.bash.

@jefferai
Copy link
Author

jefferai commented Oct 1, 2015

#9344 is the exact issue I've been having problems with. There are several mentions of released builds, for instance, #9344 (comment) and #9344 (comment)

It's not clear where CGO_ENABLED is toggled to 1 in a native build. Regardless, with CGO_ENABLED=0 exported and running all.bash, the built go binary is not static. This does work for my programs, though.

@rakyll rakyll changed the title Building a program with non-release version of Go is dynamic by default cmd/go: building a program with non-release version of Go is dynamic by default Oct 1, 2015
@ianlancetaylor
Copy link
Contributor

The comments you link to from #9344 are both out of date, as it happens. In Go 1.5 the -a option to go build does rebuild everything, including the standard library, whether you are running a release build or not.

I just tried running

CGO_ENABLED=0 bash make.bash

on a clean checkout of Go tip. It built a statically linked program version of ../bin/go.

But, you are correct: when, using that version of go, I run go env CGO_ENABLED, it prints 1, and it will link packages like os/user dynamically. I did not expect that. I'm not sure whether it is a bug or not.

@ianlancetaylor ianlancetaylor changed the title cmd/go: building a program with non-release version of Go is dynamic by default build: setting CGO_ENABLED when building does not affect built go tool Oct 2, 2015
@ianlancetaylor ianlancetaylor added this to the Go1.6 milestone Oct 2, 2015
@jefferai
Copy link
Author

jefferai commented Oct 2, 2015

OK -- I didn't realize the behavior changed again in 1.5.

I'll keep an eye on this report. Thanks for looking into it.

@rsc rsc changed the title build: setting CGO_ENABLED when building does not affect built go tool build: CGO_ENABLED=0 ./make.bash does not force default in later runs of cmd/go Dec 5, 2015
@rsc
Copy link
Contributor

rsc commented Dec 5, 2015

I am not 100% sure the requested behavior is what we want. I'm curious whether it changed or if it has always been the way it is now. I suspect the latter.

@rsc
Copy link
Contributor

rsc commented Dec 17, 2015

Looked some more. I think it has always been this way, at least back to Go 1.2: CGO_ENABLED controls now, not later. It does not bake in a default value during make.bash. Still not sure whether we want to change it.

@jefferai
Copy link
Author

@rsc I appreciate you looking. As you can see from the initial issue I referenced (and others I can point to) usage of CGO_ENABLED vs. other methods of ensuring that a built binary is static (when not cross-compiling) has been a point of confusion, and I think a lot of that has been due to the changes to e.g. -a in 1.4 and more changes in 1.5.

Feel free to close this, unless you want to keep it open as a documentation enhancement request.

@rsc
Copy link
Contributor

rsc commented Dec 17, 2015 via email

@rsc
Copy link
Contributor

rsc commented Jan 6, 2016

I think this should probably force the default but it's too late for Go 1.6.

@rsc rsc modified the milestones: Go1.7Early, Go1.6 Jan 6, 2016
@bradfitz bradfitz modified the milestones: Go1.8Early, Go1.7Early May 5, 2016
@bradfitz
Copy link
Contributor

bradfitz commented May 5, 2016

Didn't happen for 1.7 either.

@quentinmit quentinmit added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Sep 26, 2016
@rsc
Copy link
Contributor

rsc commented Oct 17, 2016

OK, let's try this and see what breaks. Will send CL.

@rsc rsc added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Oct 17, 2016
@gopherbot
Copy link

CL https://golang.org/cl/31141 mentions this issue.

mattbostock added a commit to mattbostock/timbala that referenced this issue Aug 21, 2017
Take advantage of the new multi-stage build feature in Docker to build
the application binary in a separate container to the final build
container.

The final build container is now 21MB in size versus 752MB, i.e.
2.79% of the previous size.

From quick testing, the integration tests now take 1 minute 12 seconds
versus 1 minute 47 seconds.

To avoid complications arising from dynamic linking, I've set
`CGO_ENABLED=0` to make Go compile the binary statically:
golang/go#12808 (comment)

Use the `before_install` step in `.travis.yml` to use the latest
available Docker version, since the version supported by default in
Travis (version 17.03.1-ce, build c6d412e, at the time of writing) does
not yet support multi-stage builds:
https://docs.travis-ci.com/user/docker/#Installing-a-newer-Docker-version

Multi-stage builds were introduced by version 17.06 CE:
https://blog.docker.com/2017/07/multi-stage-builds/

See:
https://docs.docker.com/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds
https://blog.alexellis.io/mutli-stage-docker-builds/
mattbostock added a commit to mattbostock/timbala that referenced this issue Aug 21, 2017
Take advantage of the new multi-stage build feature in Docker to build
the application binary in a separate container to the final build
container.

The final build container is now 21MB in size versus 752MB, i.e.
2.79% of the previous size.

From quick testing, the integration tests now take 1 minute 12 seconds
versus 1 minute 47 seconds.

To avoid complications arising from dynamic linking, I've set
`CGO_ENABLED=0` to make Go compile the binary statically:
golang/go#12808 (comment)

Use the `before_install` step in `.travis.yml` to use the latest
available Docker version, since the version supported by default in
Travis (version 17.03.1-ce, build c6d412e, at the time of writing) does
not yet support multi-stage builds:
https://docs.travis-ci.com/user/docker/#Installing-a-newer-Docker-version

Multi-stage builds were introduced by version 17.06 CE:
https://blog.docker.com/2017/07/multi-stage-builds/

See:
https://docs.docker.com/engine/userguide/eng-image/multistage-build/#use-multi-stage-builds
https://blog.alexellis.io/mutli-stage-docker-builds/
@golang golang locked and limited conversation to collaborators Oct 17, 2017
@rsc rsc removed their assignment Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

6 participants