From 3b954c5bbc826d9764febe99052fd1701073f8d8 Mon Sep 17 00:00:00 2001 From: Kent Knox Date: Wed, 27 Apr 2016 18:21:12 -0500 Subject: [PATCH] Refactored rocm-project to use amd apt-get packages repo In the docker-compose yaml file, a new target to launch rocm-project without mapping any volumes is available. This target only uses the binary bits availalbe from the amd apt-get repo. The old targets were merged into a new one, and now map the built from source bits on top of the amd apt repo data, hiding it. License changed to MIT --- LICENSE | 40 ++++++++++++++------------------ README.md | 17 +++++++++++--- docker-compose.yml.template | 33 +++++++++++++------------- roc-setup.sh | 9 +++++--- rocm-project/Dockerfile | 46 ++++++++++++++++++------------------- 5 files changed, 77 insertions(+), 68 deletions(-) diff --git a/LICENSE b/LICENSE index 3eeaa89..745f516 100644 --- a/LICENSE +++ b/LICENSE @@ -1,27 +1,21 @@ -Copyright (c) 2016, Advanced Micro Devices, Inc. -All rights reserved. +The MIT License (MIT) -Redistribution and use in source and binary forms, with or without modification, -are permitted provided that the following conditions are met: +Copyright (c) 2016 Advanced Micro Devices, Inc. -1. Redistributions of source code must retain the above copyright notice, this -list of conditions and the following disclaimer. +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: -2. Redistributions in binary form must reproduce the above copyright notice, -this list of conditions and the following disclaimer in the documentation -and/or other materials provided with the distribution. +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. -3. Neither the name of the copyright holder nor the names of its contributors -may be used to endorse or promote products derived from this software without -specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 2b0c02a..d70982b 100644 --- a/README.md +++ b/README.md @@ -20,12 +20,23 @@ Without explicit parameters, `./roc-setup` default flags are --master && --relea `./roc-setup` generates Dockerfiles to be consumed by the docker build engine. Each sub-directory of this repository corresponds to a docker 'build context' responsible for a software layer in the ROCm stack. After running the script each directory contains a generated 'Dockerfile'. The parameters to the script control which flavor of the components to build, for instance: debug builds of the /develop branches. +**What are the individual ROCm software components** + | ROC component | | |-----|-----| -| hcc-isa | the compiler that generates GPU ISA from the backend | -| hcc-hsail | the compiler that generates HSAIL IL from the backend | -| rocr | the runtime | | roct | the kernel thunk library | +| rocr | the runtime | +| hcc-hsail | the compiler that generates HSAIL IL from the backend | +| hcc-lc | the compiler that generates GPU ISA from the backend | + +**What repo branches are built with roc-setup flags** + +|| --master | --develop | +|------|-----|-----| +|roct| master | dev | +|rocr| master | dev | +|hcc-hsail| master | develop | +|hcc-lc| testing | master | The ROCm component that can not be used in a docker image is the ROCK-Kernel-Driver[1](#ROCK). In order for the docker framework to function, **the ROCm kernel must be installed on the host machine.** This is a design constraint of docker; the linux kernel is not resident in the container. All containers share the host linux kernel, so the ROCK-Kernel-Driver component must be installed on the host linux kernel. diff --git a/docker-compose.yml.template b/docker-compose.yml.template index e0b1649..8e613b4 100644 --- a/docker-compose.yml.template +++ b/docker-compose.yml.template @@ -1,3 +1,7 @@ +# ################################################# +# Copyright (c) 2016 Advanced Micro Devices, Inc. +# ################################################# + version: '2' services: # The following defines data-only containers @@ -51,13 +55,24 @@ services: # The following defines application containers, which depend on the data-only # containers defiend above to provide their software layers # These should be run with `docker-compose run --rm ` - rocm-project-isa: + rocm: build: context: ./rocm-project dockerfile: Dockerfile devices: - "/dev/kfd" - image: rocm/rocm-project-isa + image: rocm/rocm + # volumes is used to map host directories into the container + volumes: + - ~:/usr/local/src/host-home + + rocm-source: + build: + context: ./rocm-project + dockerfile: Dockerfile + devices: + - "/dev/kfd" + image: rocm/rocm-source # volumes is used to map host directories into the container volumes: - ~:/usr/local/src/host-home @@ -65,18 +80,4 @@ services: - roct:ro - rocr:ro - hcc-lc:ro - - rocm-project-hsail: - build: - context: ./rocm-project - dockerfile: Dockerfile - devices: - - "/dev/kfd" - image: rocm/rocm-project-hsail - # volumes is used to map host directories into the container - volumes: - - ~:/usr/local/src/host-home - volumes_from: - - roct:ro - - rocr:ro - hcc-hsail:ro diff --git a/roc-setup.sh b/roc-setup.sh index 0d1bcc2..4ae766e 100755 --- a/roc-setup.sh +++ b/roc-setup.sh @@ -1,3 +1,6 @@ +# ################################################# +# Copyright (c) 2016 Advanced Micro Devices, Inc. +# ################################################# #!/usr/bin/env bash # Set reasonable defaults for dockerfile builds @@ -65,18 +68,18 @@ while :; do shift done -# hcc-hsail does not have a develop branch -export repo_branch_hcc_hsail="master" - # hcc-lc conforms to a non git-flow naming scheme, 'master' changes the most export repo_branch_hcc_lc= +export repo_branch_hcc_hsail= export repo_branch= if [ -n "${build_master}" ]; then repo_branch="master" + repo_branch_hcc_hsail="master" repo_branch_hcc_lc="testing" else repo_branch="dev" + repo_branch_hcc_hsail="develop" repo_branch_hcc_lc="master" fi diff --git a/rocm-project/Dockerfile b/rocm-project/Dockerfile index fa23eb4..7981db8 100644 --- a/rocm-project/Dockerfile +++ b/rocm-project/Dockerfile @@ -1,37 +1,35 @@ -# This is a template dockerfile meant to be personalized for application development. +# This dockerfile meant to be personalized for application development. # Copy this dockerfile into a new build context (directory) and modify to taste. -# After a new build context is created, build with -# `docker build -t / .` - -# Run the container to start up a development environment. Optionally, map host -# directories into the container with -v for development convenience, which is -# especially nice for source code to persist after the container closes. -# The following maps the host directory into the container 'read only' +# It is recommended to control docker containers through 'docker-compose' https://docs.docker.com/compose/ +# Docker compose depends on a .yml file to control container sets +# roc-setup.sh can generate a useful docker-compose .yml file +# `docker-compose run --rm ` +# If running the container manually through the docker command-line, the following is an example # 'docker run -it --rm -v [host/directory]:[container/directory]:ro /'. -# Example: 'docker run -it --rm -v ~/src/my-hcc-project:/root/my-hcc-project:ro kknox/my-hcc-project'. # The application container can inherit from any other OS container FROM ubuntu:14.04 MAINTAINER Kent Knox -# Change WORKPATH to a location where -ENV WORKPATH /opt/my-rocm-project - -# The working directory is meant to be where build files are generated -WORKDIR ${WORKPATH} - # Default to a login shell ENTRYPOINT ["/bin/bash"] CMD ["-l"] -# Initialize the image to install common and recommended dev tools +# Initialize the image to install recommended dev tools and ROCm packages +RUN DEBIAN_FRONTEND=noninteractive apt-get install -y curl + +# The following imports a key to the external AMD packages repository to make it trusted +RUN curl -sL http://packages.amd.com/rocm/apt/debian/rocm.gpg.key | apt-key add - && \ + sh -c 'echo deb [arch=amd64] http://packages.amd.com/rocm/apt/debian/ trusty main > /etc/apt/sources.list.d/rocm.list' + +# Initialize the image to install recommended dev tools and ROCm packages RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ build-essential \ clang-3.5 \ libelf1 \ - curl \ + rocm \ git \ gdb \ valgrind \ @@ -40,12 +38,14 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ apt-get clean && \ rm -rf /var/lib/apt/lists/* +# This is meant to be an interactive container; create a non-root user +RUN useradd --create-home --shell /bin/bash rocm-user +USER rocm-user +WORKDIR /home/rocm-user + # The following attempts to create a nice command line developer environment # 1. Sets up an enhanced command line dev environment within VIM # 2. Aliases GDB to use its native TUI mode by default -RUN curl https://j.mp/spf13-vim3 -L | bash && \ - echo "\nalias gdb='gdb --tui'" >> ~/.bashrc && \ - echo "\nsource /opt/roct/setenv-roct.sh" >> ~/.bashrc && \ - echo "\nsource /opt/hsa/setenv-rocr.sh" >> ~/.bashrc && \ - echo "\nsource /opt/hcc/setenv-hcc.sh" >> ~/.bashrc && \ - mkdir -p ${WORKPATH} +RUN curl -sL https://j.mp/spf13-vim3 | bash && \ + echo "\n PATH=${PATH}:/opt/rocm/bin\n" >> ~/.bashrc && \ + echo "alias gdb='gdb --tui'\n" >> ~/.bashrc