Skip to content

Use shared mounts instead of containerized #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
huggsboson wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Use shared mounts instead of containerized #2

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions kube-up.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ if [ $? != 0 ]; then
fi

if [[ ! $(docker version --format {{.Server.Version}}) == "1.10.3" ]]; then
echo "Warning: You should be running docker 1.10.3"
echo "Error: You should be running docker 1.10.3"
exit 1
fi

echo "Setting up kubectl context"
Expand All @@ -38,11 +39,17 @@ fi

echo "Cleaning up last kubernetes run (may ask for sudo)"
if command -v docker-machine >/dev/null 2>&1; then
docker-machine ssh $DOCKER_MACHINE_NAME "mount | grep -o 'on /var/lib/kubelet.* type' | cut -c 4- | rev | cut -c 6- | rev | xargs --no-run-if-empty sudo umount"
docker-machine ssh $DOCKER_MACHINE_NAME "mount | grep -o 'on /var/lib/kubelet.* type' | cut -c 4- | rev | cut -c 6- | rev | sort -r | xargs --no-run-if-empty sudo umount"
docker-machine ssh $DOCKER_MACHINE_NAME "sudo rm -Rf /var/lib/kubelet"
docker-machine ssh $DOCKER_MACHINE_NAME "sudo mkdir -p /var/lib/kubelet"
docker-machine ssh $DOCKER_MACHINE_NAME "sudo mount --bind /var/lib/kubelet /var/lib/kubelet"

@kunalparmar kunalparmar Jun 30, 2016

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does binding /var/lib/kubelet to itself do?

@huggsboson huggsboson Jun 30, 2016

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sets up a bind mount which I can then change to shared below. Basically the overall context is that since kubelet itself is in a container and it's trying to share emptyDirs, secrets, configMaps with other containers in its /var/lib/kubelet dir (which is where those things go) then they need to be in a mount marked as shared so that it can be shared across 2 containers. (kubelet and the container trying to use those mounts).

docker-machine ssh $DOCKER_MACHINE_NAME "sudo mount --make-shared /var/lib/kubelet"
else
mount | grep -o 'on /var/lib/kubelet.* type' | cut -c 4- | rev | cut -c 6- | rev | xargs --no-run-if-empty sudo umount
mount | grep -o 'on /var/lib/kubelet.* type' | cut -c 4- | rev | cut -c 6- | rev | sort -r | xargs --no-run-if-empty sudo umount
sudo rm -Rf /var/lib/kubelet
sudo mkdir -p /var/lib/kubelet
sudo mount --bind /var/lib/kubelet /var/lib/kubelet
sudo mount --make-shared /var/lib/kubelet
fi

if [[ -f "$HOME/.docker/config.json" ]]; then
Expand All @@ -67,15 +74,14 @@ docker run \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:rw \
--volume=/var/run:/var/run:rw \
--volume=/var/lib/kubelet:/var/lib/kubelet:rw \
--volume=/var/lib/kubelet:/var/lib/kubelet:shared \
$private_repo_creds_mount \

@kunalparmar kunalparmar Jun 30, 2016

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does running kubelet as a container cause problem?

@huggsboson huggsboson Jun 30, 2016

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

containerized was they way they did nsenter's (which they consider an abomination) to emulate the thing that is now being down through mounting as shared above.

--net=host \
--pid=host \
--privileged=true \
-d \
gcr.io/google_containers/hyperkube-amd64:v1.2.0 \
/hyperkube kubelet \
--containerized \
--hostname-override="127.0.0.1" \
--address="0.0.0.0" \
--api-servers=http://localhost:8080 \
Expand Down