Study notes
VS code is a great tool to create and maintain applications, but it is as well a great too to manage Azure cloud resources fully integrated with GitHub.
It has a massive collection of extensions.
Last but very important: you can manage Kubernetes cluster from Docker and Azure. That the cherry on top so far because any deep learning experiment can be developed, tested and debugged locally.
PowerShell commands history
C:UsersUSER_NAMEAppDataRoamingMicrosoftWindowsPowerShellPSReadline
How to use it in Azure Machine learning - experiments (Data science)?
Requirements:
Install:
- Visual studio code
- Visual studio code extensions - mandatory:
- Python
- Jupiter
- Azure Machine Learning
- Visual studio code extensions - useful:
- Polyglot Notebooks - https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.dotnet-interactive-vscode
- Remote SSH
Note before reading further.
If you have a fresh new install of VS Code, only on python kernel available, installed through VS Code then all may work somehow, I mean all except torch and Azure SDK
Otherwise:
Install Anaconda
Add all packages you need via conda (Anaconda navigator is really nice; need a good computer)
In VS Code select Python inconda environment
You will have all you ever need for any experiment and no problem with dependecies and pacjkages update.In images (Docker or ACI) just keep simple; one pyton version and only packages you need via YAML file.
Upgrade pip (pip is a packages utility for python.
# In terminal run
python -m pip install -U pip
If you have not installed
# run in terminal or command windows prompt (as administrator)
# replace 19.1.1 with the last stable version, check https://pypi.org/project/pip/
python -m pip install downloads/pip-19.1.1-py2.py3-none-any.whl
#or
python -m pip install downloads/pip-19.1.1.tar.gz
#or
https://files.pythonhosted.org/packages/cb/28/91f26bd088ce8e22169032100d4260614fc3da435025ff389ef1d396a433/pip-20.2.4-py2.py3-none-any.whl -O ~/pip20.2.4 then do python -m pip install ~/pip20.2.4
# replace 19.1.1 with the last stable version, check https://pypi.org/project/pip/
python -m pip install downloads/pip-19.1.1-py2.py3-none-any.whl
#or
python -m pip install downloads/pip-19.1.1.tar.gz
#or
https://files.pythonhosted.org/packages/cb/28/91f26bd088ce8e22169032100d4260614fc3da435025ff389ef1d396a433/pip-20.2.4-py2.py3-none-any.whl -O ~/pip20.2.4 then do python -m pip install ~/pip20.2.4
Install basic python libraries used in Data Science (Azure ML)
pip install pandas
pip install numpy
pip install matplotlib
pip install scipy
pip install scikit-learn
Work with deep learning:
pip install torchvision
# if not, you may try
# or !pip install torch==1.9.0+cpu torchvision==0.10.0+cpu torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html (triky stuff with versions)
pip3 install torchvision
Just in case there are problems try this:
pip install --upgrade setuptools
Install python SDK
pip install azureml-core
pip install azure-ai-ml
Check:
pip list
Create first jupyter note.
Create a new file and give extension .ipynb
or
Use Command Palette (Ctrl+Shift+P) and run.
Create: New Jupyter Notebook
Write a test (python code) and run it.
To get data from Git or whatever other place you need wget utility.
In Linux you already have it.
On Windows and Mac OS:
Download it from https://www.gnu.org/software/wget/
Copy it (wget.exe) where you need ( do not leave it in downloads folder)
For example C:Wget or C:Program FilesWget (you must create Wget folder.
Add C:Wget in the Environment variable.
There are plenty of tutorials on the net, here there are two:
Windows - Command prompt: Windows CMD: PATH Variable - Add To PATH - Echo PATH - ShellHacks
Windows UI:How to Add to Windows PATH Environment Variable (helpdeskgeek.com)
Git / GitHub
Gits is part of anyone who run experiments.
Now VS Code fully integrate Git in UI
Install Git extension.
Basic operation:
1. Stage changes -> Click on + (right to added/changed/deleted file or on top line (Changes)
Write name of "Commit." and click "Commit"
2. Create Branch - 3 dots (very top)
Example "My New Branch"
3. Merge a branch to master:
- Click on 3 dots (very top)
- Select [Checkout to] and then the branch TO - master in this case
- Again 3 dots
- Click on [Branch] ->[Merge Branch] and then branch FROM - My New Branch, in this case
- Again 3 dots
- Select [Pull, Push] -> [Push]
Resources:
Working with Jupyter Notebooks in Visual Studio Code
Doing Data Science in Visual Studio Code
torch.nn — PyTorch 1.13 documentation