Monday, July 04, 2011

Git: Branching and Merging

Git allows us to maintain multiple branches of development.  This is common and consider a must to develop with git.  You may switch between branches to enjoy isolate development workflow.

To ease the explanation, we create a bare repository as origin and 2 cloned repository from origin.

$ mkdir project.git 
$ cd project.git/ 
$ git init --bare 
Initialized empty Git repository in /tmp/test.branch/project.git/ 
$ cd .. 
$ git clone project.git clone1 
Initialized empty Git repository in /tmp/test.branch/clone1/.git/ 
warning: You appear to have cloned an empty repository. 
$ cd clone1 
$ echo "This is readm" > readme 
$ git add readme 
$ git commit -m "first commit" 
[master (root-commit) 8b60032] first commit 
1 files changed, 1 insertions(+), 0 deletions(-) 
create mode 100644 readme 
$ git push 
No refs in common and none specified; doing nothing. 
Perhaps you should specify a branch such as 'master'. 
fatal: The remote end hung up unexpectedly 
error: failed to push some refs to '/tmp/test.branch/project.git' 
$ git push origin master 
Counting objects: 3, done. 
Writing objects: 100% (3/3), 215 bytes, done. 
Total 3 (delta 0), reused 0 (delta 0) 
Unpacking objects: 100% (3/3), done. 
To /tmp/test.branch/project.git 
* [new branch]      master –> master 
$ cd .. 
$ git clone project.git clone2 
Initialized empty Git repository in /tmp/test.branch/clone2/.git/ 
$ ls -a clone2 
.  ..  .git  readme

Create git branch

Let’s create a new branch “newfeature1” in clone1:

$ cd clone1 
$ git branch 
* master 
$ git branch newfeature1
$ git branch 
* master 
  newfeature1 
$ git checkout newfeature1 
Switched to branch 'newfeature1' 
$ git branch 
  master 
* newfeature1

The repository is now point to branch newfeature1.  Add a new file to the repository and commit changes:

$ echo "This is new feature" > feature1 
$ git add feature1 
$ git commit -m "Commit new feature" 
[newfeature 399cffe] Commit new feature 
1 files changed, 1 insertions(+), 0 deletions(-) 
create mode 100644 feature1

Switch branch

Continue from above example.  Let’s switch between branches and see what happen to the working directory:

$ git branch 
  master 
* newfeature1 
$ ls -a 
.  ..  .git  feature1  readme 
$ git checkout master 
Switched to branch 'master' 
$ ls -a 
.  ..  .git  readme

When switch branch from “newfeature1” to “master”, file “feature1” that was committed in “newfeature1” won’t appear in branch “master”.

List branches in repository

To list existing local branches:

$ git branch 
* master 
  newfeature1

Active branch is indicated by * in front.

To list local and remote tracked branch

$ git branch -a 
* master 
  newfeature1 
  remotes/origin/master

To list remote tracked branch only:

$ git branch -r 
  origin/master

Merge with Fast Forward

Assume we perform few commits into branch “newfeature1”:

$ echo "changes #1" >> feature1 
$ git commit -a -m "commit changes #1" 
[newfeature1 fc6a399] commit changes #1 
1 files changed, 1 insertions(+), 0 deletions(-) 
$ echo "changes #2" >> feature1 
$ git commit -a -m "commit changes #2" 
[newfeature1 1ea0fda] commit changes #2 
1 files changed, 1 insertions(+), 0 deletions(-) 
$ git log --oneline --graph 
* 1ea0fda commit changes #2 
* fc6a399 commit changes #1 
* 048ce34 Commit new feature 1 
* 8b60032 first commit

Perform a fast forward merge into master:

$ git checkout master 
Switched to branch 'master' 
$ git merge newfeature1 
Updating 8b60032..1ea0fda 
Fast-forward 
feature1 |    3 +++ 
1 files changed, 3 insertions(+), 0 deletions(-) 
create mode 100644 feature1 
$ git log --oneline --graph 
* 1ea0fda commit changes #2 
* fc6a399 commit changes #1 
* 048ce34 Commit new feature 1 
* 8b60032 first commit

All commit changes from newfeature1 is now merge into master.

Merge without Fast Forward

An advantage with no fast forward merge preserve the branch commit history:

$ git log --oneline --graph 
* 8b60032 first commit 
$ git merge --no-ff newfeature1 
Merge made by recursive. 
feature1 |    3 +++ 
1 files changed, 3 insertions(+), 0 deletions(-) 
create mode 100644 feature1 
$ git log --oneline --graph 
*   4725f13 Merge branch 'newfeature1' 
|\ 
| * 1ea0fda commit changes #2 
| * fc6a399 commit changes #1 
| * 048ce34 Commit new feature 1 
|/ 
* 8b60032 first commit

Push Branch to remote

So far all branches are stored in local repository only.  You may share the branch with others by pushing it to origin:

$ git branch -a 
* master 
  newfeature1 
  remotes/origin/master 
$ git checkout newfeature1 
Switched to branch 'newfeature1' 
$ git push origin HEAD 
Total 0 (delta 0), reused 0 (delta 0) 
To /tmp/test.branch/project.git 
* [new branch]      HEAD -> newfeature1 
$ git branch -r 
  origin/master 
  origin/newfeature1

Now we create more branch and push to origin:

$ git branch newfeature2 master 
$ git checkout newfeature2 
Switched to branch 'newfeature2' 
$ echo "Feature 2" > feature2 
$ git branch 
  feature3 
  master 
  newfeature1 
* newfeature2 
$ git add feature2 
$ git commit -m "commit feature 2" 
[newfeature2 ef39e45] commit feature 2 
1 files changed, 1 insertions(+), 0 deletions(-) 
create mode 100644 feature2 
$ git push origin HEAD 
Total 0 (delta 0), reused 0 (delta 0) 
To /tmp/test.branch/project.git 
* [new branch]      HEAD –> newfeature2

$ git branch newfeature3 master 
$ git checkout newfeature3 
Switched to branch 'newfeat 
$ echo "Feature 3" > feature3 
$ git add feature3 
$ git commit -m "commit feature 3" 
[newfeature3 ec1da8d] commit feature 3 
1 files changed, 1 insertions(+), 0 deletions(-) 
create mode 100644 feature3 
$ git push origin HEAD 
Counting objects: 7, done. 
Compressing objects: 100% (4/4), done. 
Writing objects: 100% (6/6), 512 bytes, done. 
Total 6 (delta 1), reused 0 (delta 0) 
Unpacking objects: 100% (6/6), done. 
To /tmp/test.branch/project.git 
* [new branch]      HEAD –> newfeature3

We have these branches now both in local and origin repositories:

$ git branch -a 
  master 
  newfeature1 
* newfeature2 
  newfeature3 
  remotes/origin/master 
  remotes/origin/newfeature1 
  remotes/origin/newfeature2 
  remotes/origin/newfeature3

Fetch Remote Branch

Let’s switch to clone2:

$ cd clone2 
$ git branch -a 
* master 
  remotes/origin/HEAD -> origin/master 
  remotes/origin/master 
$ git remote show origin 
* remote origin 
  Fetch URL: /tmp/test.branch/project.git 
  Push  URL: /tmp/test.branch/project.git 
  HEAD branch: master 
  Remote branches: 
    master      tracked 
    newfeature1 new (next fetch will store in remotes/origin) 
    newfeature2 new (next fetch will store in remotes/origin) 
    newfeature3 new (next fetch will store in remotes/origin) 
  Local branch configured for 'git pull': 
    master merges with remote master 
  Local ref configured for 'git push': 
    master pushes to master (up to date)

Notice that there are 3 new branches: newfeature1, newfeature2 and newfeature3 in origin.

Now fetch from origin,

$ git fetch 
From /tmp/test.branch/project 
* [new branch]      newfeature1 -> origin/newfeature1 
* [new branch]      newfeature2 -> origin/newfeature2 
* [new branch]      newfeature3 -> origin/newfeature3 
$ git branch -a 
* master 
  remotes/origin/HEAD -> origin/master 
  remotes/origin/master 
  remotes/origin/newfeature1 
  remotes/origin/newfeature2 
  remotes/origin/newfeature3 
$ git remote show origin 
* remote origin 
  Fetch URL: /tmp/test.branch/project.git 
  Push  URL: /tmp/test.branch/project.git 
  HEAD branch: master 
  Remote branches: 
    master      tracked 
    newfeature1 tracked 
    newfeature2 tracked 
    newfeature3 tracked 
  Local branch configured for 'git pull': 
    master merges with remote master 
  Local ref configured for 'git push': 
    master pushes to master (up to date)

To have a look on the remote branch newfeature2, use git checkout:

$ git checkout origin/newfeature2 
Note: checking out 'origin/newfeature2'.

You are in 'detached HEAD' state. You can look around, make experimental 
changes and commit them, and you can discard any commits you make in this 
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may 
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at c47ec36... This is new feature 2 
$ ls -a 
.  ..  .git  feature2  readme

The checkout origin/newfeature2 will switch local repository to “detached HEAD”.  A detached HEAD doesn’t belongs to any branch:

$ git branch -a 
* (no branch) 
  master 
  remotes/origin/master 
  remotes/origin/newfeature1 
  remotes/origin/newfeature2 
  remotes/origin/newfeature3

You shouldn’t commit any changes that doesn’t belong to a named branch or else others will have problems merging with the changes.  Instead you should name a branch or checkout local branch before commit any changes:

$ git checkout newfeature2
Branch newfeature2 set up to track remote branch newfeature2 from origin.
Switched to a new branch 'newfeature2'
$ git branch -a
  master
* newfeature2
  remotes/origin/master
  remotes/origin/newfeature1
  remotes/origin/newfeature2
  remotes/origin/newfeature3

Delete local branch

For branch that you never push to origin, you may delete it as follow:

$ git branch
  master
* newfeature1
$ git branch -d newfeature1
error: Cannot delete the branch 'newfeature1' which you are currently on.

Active branch is not allow to delete.  You may switch to other branch first

$ git checkout master
Switched to branch 'master'
$ git branch -d newfeature1
error: The branch 'newfeature1' is not fully merged.
If you are sure you want to delete it, run 'git branch -D newfeature1'.
$ git branch -D newfeature1
Deleted branch newfeature1 (was 201bb22).
$ git branch
* master

You can’t delete branch using “-d” if it is not merged with others.  To force delete, use “-D” option.

Delete remote tracked branch

Assume the repository has the following branches:

$ git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/newfeature1
  remotes/origin/newfeature2
  remotes/origin/newfeature3
$ git remote show origin
* remote origin
  Fetch URL: /tmp/test.branch/project.git
  Push  URL: /tmp/test.branch/project.git
  HEAD branch: master
  Remote branches:
    master      tracked
    newfeature1 tracked
    newfeature2 tracked
    newfeature3 tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (up to date)

Delete a remote tracked branch:

$ git branch -d -r origin/newfeature3
Deleted remote branch origin/newfeature3 (was ec1da8d).
$ git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/newfeature1
  remotes/origin/newfeature2
$ git remote show origin
* remote origin
  Fetch URL: /tmp/test.branch/project.git
  Push  URL: /tmp/test.branch/project.git
  HEAD branch: master
  Remote branches:
    master      tracked
    newfeature1 tracked
    newfeature2 tracked
    newfeature3 new (next fetch will store in remotes/origin)
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (up to date)

Git: Repository

Introduction

Git repository is a storage area that keep the project’s git version control information including logs, commits, version and etc. (a.k.a. repository meta data).  The repository stores in a hidden folder name .git.  There are 2 types of git repositories: bare repository and non-bare repository.

Bare repository

Bare repository is a folder that keep only git’s meta data.  It doesn’t contain project working tree.  In short, it stores all the .git folders and files directly in the folder.

Bare repository always act as a shared repository at runtime.  It has similar role in centralized version control system: the central repository in version control server.  It always act as an origin repository of many non-bare repositories of development task.

Non-bare repository

Non-bare repository or working repository is a storage folder that contain .git and working tree.  It is a repository that used in programmer’s workstation for daily work.  Most non-bare repository are usually clone from bare repository.

Create a new repository

It is common to start a repository as non bare repository:

$ git init ~/myproj
Initialized empty Git repository in /home/usr/myproj/.git/
$ cd ~/myproj
$ ls -a . .. .git $ echo my first project > readme $ git add readme $ git commit -m "My first commit" [master (root-commit) 784a3ce] My first commit Committer: root <root@revision-control.(none)> 1 files changed, 1 insertions(+), 0 deletions(-) create mode 100644 readme

A new repository has created.  Obviously, this is a non-bare repository.  It contains a .git folder a working tree.  The working tree has a file of name “readme”.

GitFaq states that

A quick rule of thumb is to never push into a repository that has a work tree attached to it, until you know what you are doing.

Thus, we shouldn’t clone the non-bare repository as origin of our working repository in development stage.  We should first make the non-bare repository to bare repository for other to clone;

$ cd ~ 
$ git clone --bare myproj myproj.git
Initialized empty Git repository in /tmp/myproj.git/ 
$ ls -a 
.   .webmin     auth    loggerhead-cache-Hpo6UL  myproj.git 
..  README.txt  config  myproj                   servers

Now, myproj.git is a bare repository of the project.  .git folder isn’t exist in bare repository.  Instead, all contents of .git stores in project folder directly.

What about myproj now?  It doesn’t know myproj.git.  It has no relation with myproj.git now.  If we continue working on myproj.  All commit works have no way to sync (push/pull) with bare repository (myproj.git).  Perform this to make myproj.git as origin of myproj.git:

$ cd ~/myproj
$ git remote add origin ~/myproj.git // for git push $ git config branch.master.remote origin // for git pull $ git config branch.master.merge refs/heads/master // for git pull

The myproj now become a clone of myproj.git.  myproj.git is origin of myproj.  You may push your commits into myproj.git as usual.  It has same working behaviour as other new clone from myproj.git.

Alternatively, you may delete myproj folder and clone from myproj.git:

$ rm –fr ~/myproj 
$ git clone ~/myproj.git ~/myproj

You may also set a default pull behaviour globally if your new project always pull from a origin branch:

$ git config --global branch.master.remote origin
$ git config --global branch.master.merge refs/heads/master

Create a bare repository

Do this If you want to start a bare repository directly:

$ git init -bare myproj.git

A bare repository is created and initialize, you may clone a new working repository from it.

Create a shared repository for team work

To create a git repository that may share for team environment accessed via SSH or other means, use something like:

$ git init --bare --shared=group project.git
Initialized empty shared Git repository in /srv/repos/git/project.git/

List the directory with attributes and notice the group write and setuid bit is on:

$ ls –l
drwxrwsr-x 7 root root 4096 Jul 7 19:27 project.git

Enable write access for developer group:

$ chgrp developer -R project.git
$ ls -l
drwxrwsr-x 7 root developer 4096 Jul 7 19:27 project.git

Share existing repository for team work

To share an existing git repository, try this:

# ls -l
drwxr-xr-x 7 root root 4096 Jul 7 15:17 project.git

Change group of repository:

# chgrp -R developer project.git
# ls -l
drwxr-xr-x 7 root developer 4096 Jul 7 15:17 project.git

Change directories permission:

# find project.git -type d |xargs chmod g+wxs
# ls -l
drwxrwsr-x 7 root developer 4096 Jul 7 15:17 project.git

Reference

  1. Stackoverflow: How do you get git to always pull from a specific branch?
  2. GitSubmoduleTutorial
  3. Git Tip of the Week: Setting up a shared repository. url: http://alblue.bandlem.com/2011/03/git-tip-of-week-setting-up-shared.html

Version Control: Git

Introduction

Git is a relatively young but powerful open source version control system compare to CVS, SVN and others.  It is famous in Linux communities.  The Linux kernel source code is using git as version control now.  More Git tools is available for Window users now.

Git doesn’t has a centralized version control server/repository like CVS or SVN.  Instead, the repository exist in each developer’s machine.  Once you have git system installed, you may start your project to enjoy git version control locally or offline.  You may carry all version control operations like commit, revert, branching, tagging and etc. in your standalone PC.  All your git work is stored in a hidden folder .git of your project.

You may however, share your repository with collaborators in your team by cloning your git repository.  To share work in team, you may push and pull with each other’s git repository.  You may also select a git repository in the team as a origin repository for everyone.  So you have a centralized control of your version control commits.  You may customize the collaboration workflow that is comfortable for your teams.  Git will suit your needs in various dimension.

Basic Git operation

init

Initialize a git repository, whether it is bare or working repository

clone

Clone repository to share repository in team work. The source repository will automatically become origin of clone repository.  You may clone local or remote repository of various source like ssh, git protocol, http or local.

add

Add changes to staging area and waiting for commit in later stage

commit

Commit all changes added to staging area

push

Push all committed snapshot to another git repository

pull

Pull all update committed snapshot from other git repository

Git Branches

Unlike other version control system, branching in Git is common and cheap operation in day to day development works.  Git doesn’t encourage developers to work with master branch directly.  Instead, all development work should work in new branch.  And merge to master once complete.

Git Best Practices

Two things you should never do in git:

  • NEVER force a push
    If you find yourself in a situation where your changes can't be pushed upstream, something is wrong. Contact another developer for help tracking down the problem.
  • NEVER rebase a branch that you pushed, or that you pulled from another person
    Rebasing published branches can lead to duplicate commits in the shared repository.

In general, the preferred workflow is:

  • create a branch from master, check it out, do your work
  • test and commit your changes
  • optionally push your branch up to the remote repository (origin) OR
  • optionally rebase your branch to master (if your changes are unpublished)
  • checkout master, make sure it's up-to-date with upstream changes
  • merge your branch into master
  • test again (and again)
  • push your local copy of master up to the remote repository master (origin/master)
  • delete your branch (and remotely, too, if you published it)

Sunday, June 19, 2011

VMware vSphere Hypervisor

Introduction

VMware vSphere Hypervisor is a bare metal hypervisor (a.k.a. type 1, native hypervisor) that run directly on host’s hardware.  It has a management interface for users to create, manage and configure virtual machines run on top of it.  VMware vSphere Hypervisor is one of the free product available to communities who like to run virtual machines natively.  A common usage for Type 1 hypervisors is deploying backend services for group or cloud applications.

An alternate virtual machine deployment strategy is running type 2 (or hosted) hypervisor under an OS.  Some examples includes KVM, VMWare Workstation and VirtualBox.  Type 2 hypervisors is used mostly in personal task that need virtual machines for testing, debugging or carry out jobs temporary without interrupting host OS.

Competitor

A direct competitor of VMware vSphere Hypervisor is XEN hypervisor.  XEN is an open source product.  There are few XEN distributors.  To get start with XEN, try Citrix XenServer.  It doesn’t involve complicated kernel compilation and installation.  It’s deployment strategy is similar to VMware vSphere Hypervisor.

After doing few performance and deployment testing for both vSphere and XEN hypervisor.  I personally few that VMware vSphere Hypervisor outperform XEN in some aspects.  The first area I test is the virtual network speed among these 2 hypervisors.  I test a host equipped with a gigabit NIC running a FreeNAS virtual machine.  I transfer a large file from CIFS share of FreeNAS.  The result is vSphere transfer at double the rate of FreeNAS.

While this doesn’t mean vSphere is simply better than XEN.  I was running XENServer out of the box. There may be options to compile an optimized version of my own XEN in order to maximised it’s performance that could perform as good as vSphere or better.  However, It is beyond my ability at the moment.  Thus, I use VMware vSphere as my first hypervisor.

Installation

Download VMware vSphere hypervisor ISO file from VMWare site.  You may need to register first before download the product.  Use one the following deployment methods that suits your environment after finish download.

Deploy via CD-ROM

This is the most easiest method.  Burn the ISO file in CD-ROM and boot the CD on the machine.  Follow instructions on screen to finish the installation.

Deploy via USB flash drive

If you want to save the world by not wasting a CD-ROM, you may create a bootable USB flash drive from the ISO by using tools like UNetbootin.  After burn the ISO into flash drive, follow the steps to create a Kickstart script or else the installation will fail in the middle.

Change file isolinux.cfg

Open isolinux.cfg in USB flash drive and add “ks=usb” as follow:

default menu.c32
menu title VMware VMvisor Boot Menu
timeout 80

label ESXi Installer
menu label ^ESXi Installer
kernel mboot.c32
append vmkboot.gz ks=usb --- vmkernel.gz --- sys.vgz --- cim.vgz --- ienviron.vgz --- install.vgz

label ^Boot from local disk
menu label ^Boot from local disk
localboot 0x80

Create a new file ks.cfg

vmaccepteula
serialnum --esx=<Serial number from VMwre>
rootpw password
autopart --firstdisk --overwritevmfs
network --bootproto=dhcp
install usb
reboot

Click here (http://www.vmware.com/pdf/vsphere4/r41/vsp_41_esxi_i_vc_setup_guide.pdf) for more information about KickStart script.

You may now boot your machine with USB boot enable in BIOS and start installation.  Please note that KickStart installation may erase your hard drive content without any warning prompt.  Use it with care.

Deploy via PXE

This is my favourite deployment strategy as all task is done via network.  You should have your PXE solution ready in your environment.

Follow the steps to prepare for PXE installation:

  1. Mount the vSphere hypervisor installation ISO file
  2. Copy all files and folders from ISO mount to TFTPBOOT folder. e.g.: /var/lib/tftpboot/vsphere.410/
  3. You may share the above folder in NFS, HTTP or FTP to use in later stage.
  4. Create a PXE entry in pxe configuration file:
    label VMware vSphere Hypervisor
          menu label VMware vSphere Hypervisor 4.1.0.update1-348481.x86_64
          kernel vsphere.410/mboot.c32
          append vsphere.410/vmkboot.gz ks=nfs://192.168.0.2/var/lib/tftpboot/ks.cfg --- vsphere.410/vmkernel.gz --- vsphere.410/sys.vgz --- vsphere.410/cim.vgz --- vsphere.410/ienviron.vgz --- vsphere.410/install.vgz
  5. The above setting use nfs to export a kickstart file ks.cfg.  You may use HTTP or FTP to export the file too.  Just find a suitable place for your KickStart file.
  6. Create a KickStart file ks.cfg:
    vmaccepteula
    serialnum --esx=<Serial number from VMwre>
    rootpw password
    autopart --firstdisk --overwritevmfs
    network --bootproto=dhcp
    install nfs --server=192.168.0.2 --dir=/var/lib/tftpboot/vsphere.410
    reboot

  7. You may use PXE boot to start the installation now.
  8. Please note that KickStart script is always required or else the installation will fail in the middle.  Again remember that KickStart installation may erase your hard drive content without any warning prompt.  Use it with care.

Configuration

Once the VMware vSphere hypervisor is up and running.  The configuration is easy.

Identify the IP address of your hypervisor and use browser to navigate to the http://<hypervisor-ip>/.  Download the Windows GUI client utility, install and start using the hypervisor.

Access vSphere hypervisor via SSH tunneling

Add forwarded port 443 and 902  to your favorite SSH client.  For example, putty SSH tunnels configuration:

1

You may then connect to your vSphere hypervisor via localhost.

The vSphere GUI client may only access to port 443, 902 and 903.  It won’t work with classic gateway port forwarding that alter the port number for client.

There is a problem with this design.  You cannot manage more than one hypervisor via SSH tunnels at the same time as the port number may conflict with each other.  A workaround solution is create a Windows machine in the same network of your hypervisors.  Install the vSphere GUI client.  Allow access the windows machine using Remote Desktop (RDP, port 3389) from remote.

Troubleshooting

Error – The Microsoft Visual J#2.0 Second Editor installer returned error code ‘4113’

You may encounter error

The Microsoft Visual J#2.0 Second Editor installer returned error code ‘4113’

when installing VMware vSphere Client 4.1.  A possible cause is you may already have a copy of Microsoft J#2.0 x86 in your system.  To overcome the problem, uninstall  Microsoft Visual J#2.0 from control panel, re-run the VMware vSphere Client 4.1 installer again.  The installer should download an updated version of Microsoft Visual J#2.0.

How to rename a .vmdk file

A vmdk file is a virtual machine disk file.  If you have created a new HDD for a virtual machine, a new .vmdk file is crated too.  vSphere doesn’t allow user to specify file name.  The file name is automatically allocated.  In common practices, we may create a new HDD device to store data purely.  We may even move the HDD to another virtual machine from time to time.  In most situations, a meaningful name is always needed to identify such HDD .vmdk file.

A solution is using console command to rename .vmdk file.  First, enable SSH from hypervisor host.  Identify .vmdk file start from folder /vmfs/volumes and use the following command to rename:

# vmfstools –E <old-name>.vmdk <new-name>.vmdk

Please note that a .vmdk HDD device consist of 2 files: <file-name>.vmdk and <file-name>-flat.vmdk

Ignore the “-flat” file and using vmfstools command to perform the rename.  Both files will be renamed accordingly.

Wednesday, November 10, 2010

Boot Linux kernel from PXE Server

It is nice and easy if we can setup an environment to perform installation of Linux via high speed LAN. The most usual method I install Linux is to burn a small boot ISO file to CDRW and use the CDRW disc to start the installation via HTTP or FTP. I learn from PXELinux and found it is a nice option to try. To setup a Linux machine via LAN, you need the following:
  1. A DHCP server that support BOOTP
  2. TFTP Server
  3. PXELinux kernel boot file
  4. Linux installation ISO file

PXELinux

You may obtain PXE Linux Boot Kernel file from http://www.kernel.org/pub/linux/utils/boot/syslinux/

Download the version of SYSLINUX you prefer and extract the tar file.  The PXE Linux Boot Kernel file name is pxelinux.0.

You should copy the pxelinux.0 to root directory of TFTP service.

DHCP Configuration

Add the following setting to dhcpd.conf to enable BOOTP

allow booting;
allow bootp;

range dynamic-bootp 192.168.0.221 192.168.0.230;
next-server 192.168.0.2;
filename "pxelinux.0";

next-server is the server of TFTP server.

TFTP Server

# Install TFTP server

yum install tftp-server

After installation,  you may proceed to TFTP configuration.  The configuration file is located at /etc/xinetd.d/tftp

TFTP Server Configuration – Enable the service

service tftp
{
        disable = no
        socket_type             = dgram
        protocol                = udp
        wait                    = yes
        user                    = root
        server                  = /usr/sbin/in.tftpd
        server_args             = -s /var/lib/tftpboot
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

TFTP Server Configuration – Firewall

-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 69 -j ACCEPT

TFTP Server Configuration – Log activities

Log activities of TFTP server Edit file /etc/xinetd.d/tftp and add "-v" option to tftpd. The flag can be specified multiple times for even higher verbosity.  You need to restart xinetd service to activate tftp logging.  The log details for TFTP may found in /var/log/messages.

service tftp
{
        disable = no
        socket_type             = dgram
        protocol                = udp
        wait                    = yes
        user                    = root
        server                  = /usr/sbin/in.tftpd
        server_args             = -v –v –s /var/lib/tftpboot
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

TFTP Server – Restart service

# /sbin/service xinetd restart

A useful example of using the log is to see what files are requested by PXE client when serving Microsoft Windows boot loader. File names are always case sensitive in TFTP operation.

TFTP Server – File name remapping

Microsoft windows or DOS use backslash (\) for folder separator. It always cause problem in those TFTP server running on Linux OS where forward slash (/) is used as folder separator. When serving windows boot loader, we may encounter the situation of we keep the boot loader files in folders. To solve this problem, we may use the file name remapping option available in tftp service.

service tftp
{
        disable = no
        socket_type             = dgram
        protocol                = udp
        wait                    = yes
        user                    = root
        server                  = /usr/sbin/in.tftpd
        server_args             = -v -m /var/lib/tftpboot/rules -s /var/lib/tftpboot
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

# vi /var/lib/tftpboot/rules
rg \\ /         # convert backslashes to slashes (useful for windows file names)

PXELinux boot menu

You may configure PXE boot menu in tftpboot/pxelinux.cfg

DEFAULT menu.c32
PROMPT 0

LABEL FC13.x86_64
        menu label Linux Fedora Core 13 x86_64 (ftp://bee/fc13.x86_64)
        KERNEL fc13.x86_64/vmlinuz
        APPEND initrd=fc13.x86_64/initrd.img

You should copy the kernel and initrd file from ISO images/pxeboot directory to tftpboot folder.

Boot PXE from client

Once the PXE server is up and ready, you may configure the client machine for network boot.  This is usually configure via BIOS setup while start the machine.  If the PXE server is setup properly, the PXE boot menu will load and you may start using PXE boot service.

Reference

    1. http://syslinux.zytor.com/wiki/index.php/PXELINUX

    Thursday, September 23, 2010

    Linux: Increase number of loop devices

    It is a common practice to mount any iso files to file system and make it available in your network environment for usage like remote installation.  You may modify /etc/fstab to make these iso files mounted during system startup.

    Mounting iso files consume loop devices (e.g.: /etc/loop*).  Linux kernel limits the number of loop devices to 8 by default.  If you have more than 8 iso files to mount, you will encounter an error: “mount: could not find any free loop device”.

    Loop Device

    Loop devices may be deployed as a module or built with Linux kernel.  It depends various Linux Distro releases.  You may use the following command to check if the loop devices are deploy with module, if not it should have built with kernel:

    -bash-3.2$ /sbin/lsmod |grep -i loop
    loop                   16581  0

    If lsmod able to list out loop module, the loop device are deployed as module.  You may then using module configuration to increase the number of loop devices.  Or else you should use append kernel parameters to increase the number of loop devices.

    Solution: loop device is a module

    1. Un-mount all files mounted to loop device
    2. Add a line in /etc/modprobe.conf to increase number of loop devices: 
        options loop max_loop=64
    3. Run
        # /sbin/rmmod loop
      to remove the loop devices module
    4. Run
        # /sbin/modprobe loop
      to re-insert the loop devices
    5. Run
        # ls /dev/loop*
      to verify loop devices has increased
    6. Run
        # mount –a
      to re-mount static file system information in /etc/fstab

    Solution: loop device built into kernel

    The following shows an example of change the kernel parameter in grub boot file to increase the number of loop devices:

    [root@dolphin mnt]# cat /boot/grub/menu.lst
    # grub.conf generated by anaconda
    #
    # Note that you do not have to rerun grub after making changes to this file
    # NOTICE:  You have a /boot partition.  This means that
    #          all kernel and initrd paths are relative to /boot/, eg.
    #          root (hd0,0)
    #          kernel /vmlinuz-version ro root=/dev/mapper/vg_dolphin-root
    #          initrd /initrd-[generic-]version.img
    #boot=/dev/sda
    default=0
    timeout=0
    splashimage=(hd0,0)/grub/splash.xpm.gz
    hiddenmenu
    title Fedora (2.6.33.3-85.fc13.x86_64)
            root (hd0,0)
            kernel /vmlinuz-2.6.33.3-85.fc13.x86_64 ro root=/dev/mapper/vg_dolphin-root \
    rd_LVM_LV=vg_dolphin/root rd_LVM_LV=vg_dolphin/swap \
    rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 \
    SYSFONT=latarcyrheb-sun16 KEYTABLE=us \
    rhgb quiet max_loop=64 initrd /initramfs-2.6.33.3-85.fc13.x86_64.img

    A “max_loop=64” is appended into kernel parameter to increase the number of loop devices to 64.  You should reboot the Linux OS to make the changes take effects.

    Tuesday, September 21, 2010

    NFS: Network File System

    Introduction

    Network File System (NFS) is a network file system protocol originally developed by Sun Microsystems in 1984, allowing an user on a client computer to access files over a network in a manner similar to how local storage is accessed.

    Installation

    yum install nfs-utils

    To check nfs version:

    [root@example]# cat /proc/fs/nfsd/versions
    +2 +3 +4 +4.1

    Default Property

    1. Server instance name: nfs
    2. Configuration file:
      1. /etc/sysconfig/nfs
      2. /etc/exports
    3. Services:
      1. portmapper (port: 111 tcp/udp)
      2. nlockmgr (port: dynamic)
      3. nfs (port: 2049 tcp/udp)
      4. mountd (port: dynamic)
    4. Log file: /var/log/messages

    Configuration

    Configuration: /etc/sysconfig/nfs

    /etc/sysconfig/nfs consist the configuration parameters for various NFS services.  Unlike other TCP/IP service, NFS service consist of several services.  Some services’ TCP and UDP ports are dynamic allocated.  You may want to fix the port numbers or else you will in trouble when configure firewall in later stage:

    # TCP port rpc.lockd should listen on.
    LOCKD_TCPPORT=32803
    # UDP port rpc.lockd should listen on.
    LOCKD_UDPPORT=32769
    
    MOUNTD_PORT=892

    Restart the NFS service for the changes to take effect:

    service nfs restart

    You may use rpcinfo –p to check if NFS services are working:

    [root@example ~]# rpcinfo -p
       program vers proto   port  service
        100000    4   tcp    111  portmapper
        100000    3   tcp    111  portmapper
        100000    2   tcp    111  portmapper
        100000    4   udp    111  portmapper
        100000    3   udp    111  portmapper
        100000    2   udp    111  portmapper
        100024    1   udp  47618  status
        100024    1   tcp  55823  status
        100011    1   udp    875  rquotad
        100011    2   udp    875  rquotad
        100011    1   tcp    875  rquotad
        100011    2   tcp    875  rquotad
        100021    1   udp  32769  nlockmgr
        100021    3   udp  32769  nlockmgr
        100021    4   udp  32769  nlockmgr
        100021    1   tcp  32803  nlockmgr
        100021    3   tcp  32803  nlockmgr
        100021    4   tcp  32803  nlockmgr
        100003    2   tcp   2049  nfs
        100003    3   tcp   2049  nfs
        100003    4   tcp   2049  nfs
        100227    2   tcp   2049  nfs_acl
        100227    3   tcp   2049  nfs_acl
        100003    2   udp   2049  nfs
        100003    3   udp   2049  nfs
        100003    4   udp   2049  nfs
        100227    2   udp   2049  nfs_acl
        100227    3   udp   2049  nfs_acl
        100005    1   udp    892  mountd
        100005    1   tcp    892  mountd
        100005    2   udp    892  mountd
        100005    2   tcp    892  mountd
        100005    3   udp    892  mountd
        100005    3   tcp    892  mountd

    Configuration: Firewall

    You may need to allow the following ports for client access:

    -A INPUT -m state --state NEW -m tcp -p tcp --dport 2049 -j ACCEPT
    -A INPUT -m state --state NEW -m udp -p udp --dport 2049 -j ACCEPT
    -A INPUT -m state --state NEW -m tcp -p tcp --dport 32803 -j ACCEPT
    -A INPUT -m state --state NEW -m tcp -p udp --dport 32769 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 111 -j ACCEPT -A INPUT -m state --state NEW -m udp -p udp --dport 111 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 892 -j ACCEPT -A INPUT -m state --state NEW -m udp -p udp --dport 892 -j ACCEPT

    Configuration: /etc/exports

    You may specify the folder you want to export in /etc/exports:

    /data   example.com(ro)
    /home   example.com(ro)

    To check what the server exports:

    exportfs -v

    To re-export modification done in /etc/exports:

    exportfs -r

    NFS Client

    Client: mount

    The following example show how to use mount to mount the NFS export file system:

    # mount example.com:/data /mnt/data
    # mount example.com:/home /home

    Client: /etc/fstab

    You may also add the NFS mount in file system table: /etc/fstab:

    example.com:/data   /mnt/data  nfs rw 0 0
    example.com:/home   /home nfs rw 0 0
    

    Client: autofs or automount

    An example using /etc/auto.indirect to serve autofs mount:

    /mnt/data         -rw example.com:/data
    /home -rw example.com:/home

    An example of using LDAP to serve autofs mount:

    dn: cn=/mnt/data,ou=auto.indirect,ou=autofs,dc=example,dc=com
    objectClass: automount
    objectClass: top
    automountInformation: -rw example.com:/data
    cn: /mnt/data
    dn: cn=/home,ou=auto.indirect,ou=autofs,dc=example,dc=com
    objectClass: automount
    objectClass: top
    automountInformation: -rw example.com:/home
    cn: /home
    

    Reference

    1. Linux NFS. url: http://www.linux-nfs.org/
    2. Linux NFS-HOWTO. url: http://nfs.sourceforge.net/nfs-howto/

    Friday, September 17, 2010

    Windows 7: Unable to remove a printer

    We may use “Remove Device” in “Devices and Printers” window to remove a printer that no longer use.  However, there are situation where the “Remove Device” doesn’t remove the printer.  Even if we perform this action in privilege account:

    Capture

    To remove this stubborn printer from your devices, we may use the Print Management Console in Administrative Tools:

    2

    Just Select the printer to remove and use the popup menu to delete printer.  You may need to perform this in privilege mode if it doesn’t work.

    3

    Thursday, September 16, 2010

    Windows 7: What’s new

    Windows Desktop

    Snap

    • Drag the title bar of an open window to either side of the desktop to align it there, or
    • Drag it to the top of the desktop to maximize the window.
    • To expand a window vertically using Snap, drag the top edge of the window to the top of the desktop.
    • To return the window to its original size, drag the title bar of the window away from the top of the screen.

    Aero Shake

    • Just click the title bar of the window you want to keep open and drag (or shake) the window back and forth quickly, and the other open windows are minimized.
    • To restore the minimized windows, shake the open window again.
    • Press Windows logo key Picture of Windows logo key +Home to minimize all windows except for the currently active window. Press Windows logo key Picture of Windows logo key +Home again to restore all windows

    Aero Peek

    • The Show desktop button has been moved the opposite end of the taskbar from the Start button, making it easier to click or point to the button without accidentally opening the Start menu.
    • You can temporarily view or peek at the desktop by just pointing to the Show desktop button. When you point to the Show desktop button at the end of the taskbar, any open windows fade from view, revealing the desktop. To make the windows reappear, move the mouse away from the Show desktop button.
    • Press the Windows logo key Picture of Windows logo key +Spacebar to temporarily preview the desktop. To restore the desktop, release the Windows logo key Picture of Windows logo key +Spacebar.
    • To minimize open windows so that they remain minimized, click the Show desktop button, or press the Windows logo key Picture of Windows logo key +D. To restore the open windows, click the Show desktop button again or press the Windows logo key Picture of Windows logo key +D again.

    Desktop background

    • Your desktop background doesn't have to be a single picture anymore. With Windows 7, you can display a slide show of pictures, instead. Some Windows themes include a slide show, or you can create your own slide show from your personal collection of pictures.

    Start menu

    Jump Lists

    • Jump Lists are lists of recent items, such as files, folders, or websites, organized by the program you use to open them.
    • In addition to being able to open recent items using a Jump List, you can also pin favorite items to a Jump List, so that you can easily access the programs and files you use every day.

    Libraries

    • A library gathers files from different locations and displays them as a single collection, without moving them from where they’re stored. There are four default libraries (Documents, Music, Pictures, and Videos), but you can create new libraries for other collections. The Documents, Music, and Pictures libraries appear on the Start menu by default. Like other items on the Start menu, you can add or remove libraries, or customize their appearance.

    Search

    • The Start menu includes a search box that you can use to find files, folders, programs, and e-mail messages stored on your computer. When you start typing a word or phrase in the search box, the search begins automatically, and the search results temporarily fill the Start menu space above the search box.

    TaskBar

    Taskbar buttons

    • Taskbar buttons have a new look and do more than just show you which programs are running.
    • In the default view, each program appears as a single, unlabeled button—even when multiple items for a program are open—for a clean and uncluttered look. You can customize the taskbar appearance to change how buttons appear and how they group together when you have multiple items open. You can also choose to see individual buttons for each open file.
    • You can also rearrange and organize buttons on the taskbar, including pinned programs and running programs that aren’t pinned, so they appear in the order you prefer. To rearrange the order of buttons on the taskbar, drag a button from its current position to a different position on the taskbar. You can rearrange buttons as often as you like.

    Previewing open windows with Aero Peek

    • When you open multiple windows on the desktop, sometimes it can be a challenge to view separate windows and switch between them.
    • You can use Aero Peek to take a quick look at other open windows without clicking away from the window you are currently working on. Point your mouse at a taskbar button, and thumbnail previews of any open windows associated with that button appear above the taskbar. If you want to open a window you are previewing, just click its thumbnail. For more information.

    Pinning items

    • Pinning programs to the taskbar complements pinning programs to the Start menu, like in earlier versions of Windows. When you pin a favorite program to the taskbar, you can always see it there and easily access it with a single click. Windows 7 also includes Jump Lists, so that in addition to launching a program from the taskbar, you can now launch favorite and recent items from that program, just by clicking the same button.
    • Jump Lists are lists of recently or frequently opened items, such as files, folders, tasks, or websites, organized by the program that you use to open them. In addition to being able to open recent items using a Jump List, you can also pin favorite items to a Jump List so you can quickly get to the items that you use every day.
    • On the taskbar, Jump Lists appear for programs that you've pinned to the taskbar and programs that are currently running. You can view the Jump List for a program by right-clicking the taskbar button, or by dragging the button toward the desktop. You open items from the Jump List by clicking them.

    Notification area

    • A new way of managing the notification area on the end of the taskbar means you get fewer notifications, and the ones you get are collected in a single place in Windows.
    • In the past, the notification area could sometimes become cluttered with icons. Now, you can choose which icons appear visible at all times. And you can keep the rest of the icons on hand in an overflow area, where they’re accessible with a single mouse click.
    • Action Center is a single area that collects important notification messages about security and maintenance settings. You can review these messages later if you don’t want to be interrupted. When you click the Action Center icon Picture of Action Center icon and then click Open Action Center, you’ll see information about the things you need to take action on, and find helpful links to troubleshooters and other tools that can help fix problems.

    Viewing the desktop

    • The Show desktop button has been moved to the opposite end of the taskbar from the Start button, making it easier to click or point at the button without accidentally opening the Start menu.
    • In addition to clicking the Show desktop button to get to the desktop, you can temporarily view or peek at the desktop by just pointing your mouse at the Show desktop button, without clicking it. When you point at the Show desktop button at the end of the taskbar, any open windows fade from view, revealing the desktop. To make the windows reappear, move the mouse away from the Show desktop button.
    • This can be useful for quickly viewing desktop gadgets, or when you don’t want to minimize all open windows and then have to restore them.