Virtual machines for data science - errors and troubleshooting

Troubleshooting of errors Virtual Machine for Data Science

For Windows OS, Linux, MacOS X

Localhost url not working in browser

If you try to load machine by hitting "http://localhost:8001/" for Spark machine or "http://localhost:5002/" for Python machine in your browser, and that doesn't work, try:

"http://127.0.0.1:8001/" or "http://127.0.0.1:5002/" respectively.

The proper choice depends on the configuration of your computer.

We pre-configured the virtual machine to start the iPython notebook on port 8001/5002 by default.

If you are having problems connecting to the notebook (neither of the two links works), you should check the output of the "vagrant up" command. It may be the case that there was a conflict on your computer with a program already using port 8001/5002 and a higher port number was automatically used. If this occurs, you should use that port number instead of port 8001/5002.

And all the rest of the instructions will be the same.

http://localhost:8001/

http://localhost:5002/

http://127.0.0.1:8001/

http://127.0.0.1:8001/

Windows

Windows - First step: Installing the VirtualBox Error

If you encounter this error:

VirtualBox - Critical Error Failed to create the VirtualBox COM object. The application will now terminate.Callee RC: E_NOINTERFACE (0x80004002)

You may not have installed VirtualBox as Administrator.

You should perform the following steps:

● Uninstall VirtualBox

● Reboot your machine

● Reinstall VirtualBox by right-clicking on the installation file and selecting "Run as Administrator"

● Once the installation completes, select "Start Menu->Oracle VM VirtualBox"

● Right click on "Oracle VM VirtualBox" and select "Properties"

● Navigate to the "Compatibility" tab

● In the Compatibility mode box, click the checkbox for "Run this program in compatibility mode for:" and select "Windows XP (Service Pack 3)"

● In the Priviledge Level box, click on the checkbox for "Run this program as an administrator"

● Click on "Change settings for all users" and make sure that both the checkboxes for "Run this program in compatibility mode for:" and "Run this program as an administrator" are set.

Windows - Second step: Installing Vagrant Error

If you encounter the Windows error: “Installation Directory must be on a local hard drive”, this is a permissions error and you need to force the installer to run with administrator privileges. For help go to the next slide.

To do so:

➔ Locate the Vagrant MSI installer file that you downloaded.

➔ While holding down the Shift key on the keyboard, right-click on the Vagrant MSI, then choose Copy As Path.

➔ Go to Start > All Programs > Accessories.

➔ Right-click on Command Prompt and choose Run As Administrator. This should open a command prompt window, labeled "Administrator:".

➔ In the Command Prompt window, type msiexec /i (you need to enter a single space after "/i").

➔ Right-click in the Command Prompt window, then choose Paste. This should paste the path to the MSI file that you copied in Step 2 above.

➔ Press Enter to run the command.

Windows - Third step: Installing the VM Error

Some comments:

➔ On Microsoft Windows you may get a Windows Firewall exception popup regarding "vboxheadless". Adding an exception is optional and does not seem to impact the Virtual Machine once it is.

➔ When running "vagrant up" again, you may receive the following

◆ bsdtar: Error opening archive: Unrecognized archive format.

➔ The solution is to first use the command:

◆ vagrant box remove Spark_machine or Python_machine.

➔ Then, you can use the "vagrant up" command again.

Mac OS X

MacOSx - Third step: Installing the VM Error

When running "vagrant up" again, you may receive the following error:

bsdtar: Error opening archive: Unrecognized archive format

The solution is to first use the command:

vagrant box remove Spark_machine

or

vagrant box remove Python_machine

Then, you can use the "vagrant up --provider virtualbox" command again.

Linux OS X

Linux - Third step: Installing the VM Error

When running "vagrant up" again, you may receive the following error:


The solution is to first use the command:

vagrant box remove Spark_machine

or

vagrant box remove Python_machine

Then, you can use the "vagrant up --provider virtualbox" command again.

Miscellaneous errors

Error Cause & Resolution

VM does not start or takes a long time, and in the end, a timeout error

Cause: slowness caused by severe resource constraintsResolution: If Vagrant is timing out it might help to start VirtualBox UI first. To do that, open the VirtualBox UI. Locate the Python_macine or Spark_machine machine and start that virtual machine. Wait and ensure it is running before going back to command prompt/Terminal to run vagrant up.


Cause: possible corruption of the VM box.

Resolution: Perform the "cleanup steps" above and try the download again.

Vagrant says virtual machine couldn't be set"The name of your virtual machine couldn't be set because VirtualBox is reporting another VM with that name already exists. Most of the time, this is because of an error with VirtualBox not cleaning up properly. To fix this, verify that no VMs with that name do exist (by opening the VirtualBox GUI). If they don't, then look at the folder in the error message from VirtualBox below and remove it if there isn't any information you need in there."

Cause: possibly partial download from a previous attempt.

Resolution: Perform the "cleanup steps" above and try the download again.


Vagrant connecting... and timeout

Cause: if on Windows possibly related to Hyper-V enabled

(Windows 8)? Also check if Firewall is enabled?

Resolution: Try to disable it.

SSL ErrorSSL certificate problem: unable to get local

issuer certificate More details here: http:

//curl.haxx.se/docs/sslcerts.html curl

performs SSL certificate verification by

default, using a "bundle" of Certificate

Authority (CA) public keys (CA certs)....

Cause: Possibly a known bug with curl used by Vagrant.

Resolution: This should no longer occur with our current

settings. Please contact the instructors on piazza if you

encounter this

Mac-only Error: Mac OSX Warning message:

"Insecure world writable dir /opt/local/bin in

PATH, mode 040777", "/Users/Username/.

rvm/rubies/ruby-1.9.1- p378/bin/gem:4:

warning: Insecure world writable dir

/opt/local/bin in PATH, mode 040777"

Cause: Your user directory is world writable.

Resolution: This should be harmless. For best practices this

should be addressed by removing world writable

permissions - Open up a Terminal window and type the

following (replace {USERNAME} with your user name):

sudo chmod go-w /Users/{USERNAME}


Linux-only Error: Incorrect Vagrant

version

Cause: Make sure you are using Vagrant 1.7 or newer with

Linux. The default Vagrant version packaged with most Linux

distributions does not seem to work properly with our course

software. If you encounter any of the following error messages

it means that your Vagrant version is out of date: SSH: The

following settings shouldn't exist: insert_key VM: The box

'datascience-school/Python_machine' or ‘'datascience-

school/Spark_machine' ‘ could not be found.

Resolution: Please download the latest version from the

vagrant site: https://www.vagrantup.com/downloads.html.

Install using the command "sudo dpkg -i vagrant.deb" and re-

run "vagrant up --provider virtualbox" and it should install

correctly.

Linux-only Error: VirtualBox

complains about kernel module not

loaded on Fedora

Resolution: Run these sudo /etc/init.d/vboxdrv setup sudo

/etc/init.d/vboxdrv start.


“Installation Directory must be on a local hard drive”

This is a permissions error and you need to force the installer to

run with administrator privileges.

To do so:

1. Locate the Vagrant MSI installer file that you downloaded.

2. While holding down the Shift key on the keyboard, right-click

on the Vagrant MSI, then choose Run As Administrator.

3. Go to Start > All Programs > Accessories.

4. Right-click on Command Prompt and choose Run As

Administrator. This should open a command prompt window,

labeled "Administrator:".

5. Run the rest of the installation step in this Command Prompt

window.

Windows Firewall exception popup regarding "vboxheadless"

Resolution: Adding an exception is optional and does not seem to

impact the Virtual Machine once it is installed.


When attempting to start VirtualBox, you may encounter the VirtualBox - Critical Error

VirtualBox - Critical Error Failed to create the VirtualBox COM object.

The application will now terminate. Callee RC: E_NOINTERFACE

(0x80004002)

If you encounter this error, you may not have installed VirtualBox as

Administrator.

You should perform the following steps:

1. Uninstall VirtualBox

2. Reboot your machine

3. Reinstall VirtualBox by right-clicking on the installation file and selecting "Run as Administrator"

4. Once the installation completes, select the shortcut in "Start Menu->Oracle VM VirtualBox"

5. Right click on the shortcut "Oracle VM VirtualBox" and select "Properties"

6. Navigate to the "Compatibility" tab

7. In the Compatibility mode box, click the checkbox for "Run this program in compatibility mode for:" and select "Windows XP (Service Pack 3)"

8. In the Priviledge Level box, click on the check box for "Run this program as an administrator" Click on "Change settings for all users" and make sure that both the checkboxes for "Run this program in compatibility mode for:" and "Run this program as an administrator" are set.


E_NOINTERFACE (0x80004002)

When you type "vagrant up --provider virtualbox", if you

encounter the following error:

Bringing machine 'Spark_machine' up with 'virtualbox' provider...

There was an error while executing 'VBoxManage', a CLI used by

Vagrant for controlling VirtualBox. The command and stderr is

shown below.

Command: ["list", "hostonlyifs"] Stderr: VBoxManage.exe: error:

Failed to create the VirtualBox object! VBoxManage.exe: error:

Code E_NOINTERFACE (0x80004002) - No such interface

supported (extended info not available) VBoxManage.exe: error:

Most likely, the VirtualBox COM server is not running or failed to

start.

1. Make sure VirtualBox is installed with "Run as Administrator" -

see above for step-by-step instructions.

2. Start command prompt using "Run as Administrator" (right-click

on command prompt and select "Run as Administrator").

Vagrant error with "incompatible character encodings"

Cause: possible bug with users having non-ASCII character

username on Windows.

Resolution: try with a user with ASCII character username or set

VAGRANT_HOME http://docs.vagrantup.

com/v2/other/environmental-variables.html https://github.

com/mitchellh/vagrant/issues/2885


VirtualBox cannot start VM from VirtualBox UI, Unable to load R3 module VBoxDD.DLL (VBoxDD): GetLastError=1790 (VERR_UNRESOLVED_ERROR)

Cause: possible cause: https://www.virtualbox.org/ticket/13504?

cversion=1&cnum_hist=1 "On my system, if the following files are

patched by the Universal Theme Patcher (by DeepXW), this

consistently generates the "Unable to load R3 module" problem

even when the v4.3.26 r98988 host is installed: themeservice.dll

themeui.dll uxtheme.dll. Generally, you can ensure that these files

are not patched or corrupted by running the following command

at the DOS command prompt: sfc /scannow." The Universal

Theme Patcher is used to enable third party themes in Windows 7

and would patch the system dll causing some errors.

Vagrant error with "Could not rename the directory" ... "VERR_ALREADY_EXISTS"

Cause: incomplete download left behind and not running as

Administrator on Windows.

Resolution: Delete Spark_machine directory from the error

message and run vagrant in Command Prompt/VirtualBox UI as

Administrator

Vagrant connecting... and timeout

Cause: if on Windows possibly related to Hyper-V enabled

(Windows 8)? Also check if Firewall is enabled?

Resolution: Try disable it.


VBoxManage.exe: error: Could not

find a registered machine with UUID ...

VBoxManage.exe: error: Details: code

VBOX_E_OBJECT_NOT_FOUND

(0x80bb0001)

Cause: Windows COM registration problem when

VirtualBox/Vagrant is not installed as administrator

Resolution: re-install everything as administrator,

1. Complete "cleanup steps" here to remove everything with VM

2. Uninstall VirtualBox and Vagrant

3. Reboot, make sure everything is gone

4. Install VirtualBox as administrator (see our installation step-

by-step if you are unsure how)

5. Install Vagrant as administrator

6. Reboot

7. Vagrant up and the rest of our installation step-by-step

Full reset of machines

How to completely reset the VM and start in a clean state?

Cleanup steps:

Note: you should only follow these steps if vagrant halt or restarting your machine doesn't help.

1. In the custom directory where you have the VagrantFile, run “vagrant destroy”.

2. Open VirtualBox UI to make sure there is no machine left there; if there is, hold down the Ctrl key and click on the machine (Right click onWindows) and select Remove... to delete it (Also after that, to be sure, delete these directory manually: {your userprofile}/VirtualBox VMs/%%MACHINENAME*).

3. Run “vagrant box remove sparkmooc/base”, and run “vagrant box list”, check that there is nothing named sparkmooc.

4. Removing a .vagrant directory where the Vagrantfile is (please note the dot in front of the directory name).

5. Restart the download process by following the steps in presentation.

Thank you for attention!

Virtual Machines by datascience-school.com

http://vm.datascience-school.com

http://datascience-school.com

Data & Analytics

Virtual machines for data science - errors and troubleshooting