Powershell - the good and the bad. The story of a survivor.

Today's topic is Powershell - it's usefulness and weirdness.

It is the Microsoft super cool console which is used to provide command-line API for all of MS big products like Office, SQL Serveer, etc. It is build on top of .Net so every .Net class you write, you can take advantage of it in the command-line.

It has it's advantages:
- using objects and not strings for pipelining commands
- the .net framework
- ease-of-use - with just a few commands you can open up the Powershell world.
- very readable
- good help about_xxx articles

Let's start with the

Good Stuff

Pipelining is a powerful feature. You can pass multiple result objects from 1 command to the next one just like this:

How you delete everything inside a folder ?

Get-Item "c:\folder\" | Remove-Item -Force -Recurse #alias is dir or ls

How to see how to use a command ? What are its parameters? What does it do?

Get-Help "Get-Item" -Full  #alias is man or help 

What are the properties/methods of a process ?

Get-Process | Select-Object -First 1 | Get-Member #alias is gm

With just Get-Help (or man), Get-Member (or gm) and Get-Command (to list all available commands/functions/aliases) , you can do anything!


You'll use pipelining all the time! It's really cool!

Some nice hits are:
- filtering a collection of items by a property. Returns all folders with name ending with the string "project"

$folderList | ? { $_.Name -match ".*project" }

which is short way to write:

Foreach-Object $folder in $folderList | Where-Object {  $folder.Name -match ".*project"}

- doing a repetitive task on many objects. Display

$folderList = dir | ? {$_.GetType().Name -match ".*DirectoryInfo.*" }

$folderList | % { Write-Host "$($_.Name) - $($_.CreationTime)" }

- errors
When there is an error just get the automatic variable $Error (array of error objects) and check the last 5 errors $Error[0..5]


Main usage of Powershell is doing some quick check on something in the Microsoft infrastructure or writing a script to automate a routine tedious task like creating a report or killing all not-responding processes at the moment.
Overall system administrators will love it.
...

and here ends the good things... If you try to use it for something else, you'll hit many different obstacles:

Strange behaviour

Functions want to return something!

In fact, every function you call that returns an object, and the result is not saved in a variable, is automatically transfered as a result. OK, but when you have multiple such calls inside your function, these results are added into an array and here's the /unexpected behaviour/...

function myFunc(){

 Get-Process

 dir

}

$b = myFunc

$b.GetType()  --- #it says array

The other way to stop this from happening is to enclose the call with brackets and [void] the result like this:

 [void] (Get-Process)

The strange empty array

When comparing empty array to $null  with the comparison operator -eq (equals), doesn't return neither $true, nor $false...just ... nothing.

This is partially explained with the Powershell ability to auomatically flatten an array to its elements. When it's an empty array - the operation doesn't execute. But there's no error as well... or warning.

Try this:

@() -eq $null

and

$null -eq @()

Also if you are used to separate each function parameter with comma, you will have a hard time!

Using comma ( ,) will generate an array. Well, Powershell likes arrays a lot, and you will, too, but only after the initial struggle.


Using it for plain programming

Unfortunately, when you want to use it to code features, and even small frameworks, you hit

some obstacles like
- Visual Studio doesn't support it... at all! No other IDE with satisfying experience, too
- PowerShell ISE isn't feature-rich enough
- only suitable is Notepad++ with word completion and many macros along the way...
- no compile-time catching bugs like - unused variables, or undefined ones...

Unfinished or just plain broken features

Modules as Custom objects
As of Powershell v2, there is a new method to package commands (cmdlets) - modules (instead of snapins).

So you can import modules into the runspace or import them as CustomObject, which has
methods as the functions declared in the .psm1 file (module script file)

These CustomObjectModules are very buggy when it comes to multiple imports and generating exceptings...Trust me. The latter is very nasty...
I
magine writing a few modules which should use each other - common utils, etc. And then you have an exception. Now where exactly did the exception occur ? Try to get catch it with try-catch - the Error record contains only the row/execution line of your current module in development! Nothing about the inner exception or error stack... !! Which is not useful at all.

In order to be productive with these modules, I ended up writing my own ExceptionStackTrace building function which collected errors, line numbers, positioned the stack trace correctly, and stops when it reaches another stack trace (.Net one) which is descriptive enough...  Well, I'm sure it could have been done better, but it works stable enough and I like that it solved this problem which should have been handled by the Powershell Team!

Also nested imports fail from time to time with functions which cannot be found...
I am seriously, not going to use Modules as CustomObjects anymore... Too bad. Almost like Object-oriented programming... but with serious issues...

Remoting
While Powershell provides nice API for remote calls - Invoke-Command for executing single scriptblock and Enter-PSSession for interactive mode. They are thought quite nice, but unfortunately, the last few weeks I've come to the conclusion that they are quite fragile and I was often the observer of the following results:
- OutOfMemory Error
- Stopped execution without errors
- WinRM was not configured correctly errors pointing to articles in MSDN which weren't specific enough.


So basically here are my findings about Powershell Remoting:

- complex requirements for remote machine for the remoting to work. Not a single full guide for this.

- adding snapins remotely for machine with no Internet access - 2 minutes ! You have to turn off the Internet Options > Advanced > Security option to "Check for publisher's certificate invocation" and it will pass in seconds.

- remoting stops without any error or logs. Hard to debug. Could be missing configurations/rights, could be low memory

- slightly more complex remoting produces many errors. For example starting a new resource-hungry process on the remote machine results in OutOfMemory Error. Solved by increasing the memory per PowerShell property described at the end of the article.

- inconsistent behaviour - different errors for unchanged code

- lack of documentation for some of the features - only a few blog posts saved me a few hours/days

A few tips here:

- the remoting script you run should be as light as possible and should just execute a file/script located on the remote machine. This was you can lower remoting data transfers(logs, commands, variables, etc)

- for complex, time-consuming tasks use Invoke-Command xxxx -AsJob parameter. This way it will create invisible background thread, which will take even less resources. Actually, for my complex scenario , it proved to be the only working option...

Get-Item wsman:localhost\Shell\MaxMemoryPerShellMB 

Set-Item wsman:localhost\Shell\MaxMemoryPerShellMB 512 -Force

Conclusion:

Don't get me wrong. Using Powershell is overall an easy and straight-forward experience, but when you try something a bit more kinky and ... let's say that the 'fun' starts there.

Best regards,
Powershell-ill
Leni Kirilov

Be efficient - virtualize and automate your test infrastructure

Author: Leni Kirilov, PowerCLI QE team, VMware Bulgaria

I met my friend Michael, a software quality engineer, who is validating the functional correctness and completeness of “Cake Wizard”. Cake Wizard is an online store, consisting of a database of items for sale, accounts, online payments, etc. It is an enterprise application developed with JEE.

In the process of validating the product, Michael needed a dedicated test environment. Michael told me he created a complete set of automation tools for executing the tests, gathering the results, and reverting the environment to its initial state. He was really proud of his accomplishment. He also integrated download and installation of product builds with a Git source control system for the test cases, Jenkins/Hudson for scheduling jobs on the test run process, and a custom test runner to execute JUnit… I was impressed! Michael had done a really great job by automating so many things. But he mentioned that he had many problems with automating the process. There were even lost test run results due to bugs.

I suggested a different approach – automation & virtualization. Michael looked confused, not to say surprised. “But virtualization is used only in production systems for better load-distribution, resource utilization and cloud services… what does it have to do with automation testing?” I smiled…

Nowadays, most testing looks like this: you get a physical environment, execute tests on it, and revert it to a clean state by using images. This approach is convenient because you don’t have to configure everything all over again. But there are some negatives as well.

- When you modify the hardware configuration, you must update the image, because it’s hardware specific;

- If the hardware configuration fails, you can lose all your data and test results, which is a potential risk for the project.

- The approach doesn’t scale well because a single hardware configuration can be at most a single test environment.

Virtualization has the same advantages as images-driven test environment setup and more:

- Better utilization – you can run more than one virtual machine (VM) on a single piece of hardware as long as there are enough resources. Provide your Quality Engineers their own environment to play with!

- Revert to a previous state – cleanup is easy because VMs are designed with that in mind. You can easily debug a problem by using snapshots. They are quite fast, too.

- Hardware failure – it’s irrelevant for the virtualized infrastructure. Since VMs are just files, you can back them up, in order to minimize loss, in case of server crashes

- Pool of resources – no need to reconfigure and recreate images when you buy new hardware. You just add it to the pool of resources.

- Maintenance & automation – with minimal effort you can replicate VMs by using virtualization and the already available automation scripts. You can even create your own automation scripts!

My friend Michael heard my arguments but was still skeptical. He had already created his environment and he asked me: “What’s the easiest way to virtualize it? What are the first steps?”

Let’s see how easy we can use VMware virtualization to:
- automate setting up a test environment,
- set up the test software
 - collect test results

[Set up and configure virtual test environment]

You need virtualization infrastructure, so let’s install VMware ESXi Hypervisor. It provides tools to run and manage virtual machines. Follow these instructions

You start your virtualized test environment by creating a empty virtual machine (VM). You install the OS (Guest OS) and applications you need just once and save the result VM. You can use it to clone and thus multiply the VM as many times as you need.

Let’s install Windows Server 2008 and a JBoss Application Server 6 on our VM and name it “Manually created VM”.

Manually configuring a VM is OK only in the process of learning the VM specifics. As Quality Engineers, we want to automate as much as possible in our work.

“How do we automate a virtual test environment and what is the best tool?”

The preferred tool for virtualization automation is VMware vSphere PowerCLI. It is closely developed with the VMware infrastructure suite (called vSphere). We need only some of the many features which PowerCLI provided, to create our test environment. The product platform is Microsoft PowerShell (running on .Net) – powerful yet very user friendly CLI (command line interpreter) which provides great console commands(called cmdlets) for managing other Windows products like SQL Server, IIS, Windows, MS Office etc. Even though PowerCLI is based on Microsoft technology, you can use it to install and configure Windows, as well as Red Hat Guest OS on your VM. Let’s see it in action:

Download and install latest PowerCLI version from here: http://www.vmware.com/go/powercli

To start using PowerCLI you should connect to your local VMware ESXi Server (which you should have installed and configured beforehand).

Connect-VIServer 192.168.1.1 -User ‘administrator’ -Password ‘pass1234’

#For later processing get your VM and store it in a variable.

$vm = Get-VM –Name “Test_Environment_VM_1”

Start-VM  $vm # you can start using your VM for configuration

It’s highly recommended to install VMware Tools on your VM – it makes some of the trickier operations possible, improves VM’s performance…
Let’s get all network interface cards of the OS installed on the VM and then configure them.

# get all network cards. GuestUser is the username required for login to the GuestOS

$nics = Get-VMGuestNetworkInterface $vm -GuestUser ‘administrator’ -GuestPassword ‘pass1234′

We have dedicated the 192.168.2.* subnetwork for our test infrastructure, so let’s update the VMs IP and DNS addresses:

Set-VMGuestNetworkInterface $nics[0] -Ip 192.168.2.2 –Dns 192.168.1.2 -GuestUser ‘administrator’ -GuestPassword ‘pass1234′

If you need a more advanced configuration for your Guest OS, you can take a look at the OSCustomization cmdlets, which give you the ability to configure user accounts, active directory, workgroups as the VM is created. There are examples in the documentation.

After you finish configuring your networking, you should create a snapshot that saves the state of your VM. This is useful when you want to revert this VM to a previous state. Let’s create a snapshot for future use:

New-Snapshot –VM $vm –Name “After network configuration – 2012.02.20”

[Create environment configuration files]

It’s good to keep various data about your environment. We create a hashtable called $env1. It can store the names of virtual machines, the servers’ IP addresses, DB table names, etc.

You can use the NIC objects to build this configuration file for future test executions or to form a report. The $nics variable contains all the information you need such as IP, DNS, Subnetmask, etc:

$env1[“IpAddress”] = $nics[0].Ip

You can save the hashtable $env1 to a file and use it as configuration for your “execute test” script. This way you can easily switch between environments with various properties.

$env1 | Export-CliXml “d:\CakeWizard\Testing\Environments\env1_win2008.xml”

#to use the configuration you import it using this call:

$env1Config = Import-CliXml “d:\CakeWizard\Testing\Environments\env1_win2008.xml”

Now you can combine all these one-liners in a short script that configures VM’s networking and creates a report/config file with the current IPs that are in use.

Simple, huh?

“What about applications and integration with other systems like Source Control Repositories or test execution systems like Hudson/Jenkins?”

[Set up tests and test framework]

For even greater customization, you can execute custom scripts in the Guest OS! By copying files to the Guest OS and executing them, you can automate the following action:
- downloading and installing the software under testing;
- syncing sources with tests and frameworks;
- preparing reports;
- managing Microsoft products via their PowerShell API;
- pretty much everything you can think of!

You can copy different executable files, ZIP files with scripts, DLLs… everything you want, to the VM’s local file system. Let’s sync the tests from our source control repository. I’ve written a short script to create workspace and sync everything with it.

Copy-VMGuestFile -LocalToGuest -VM $vm -Source “d:\CakeWizard\tools\sources_sync.bat” –Destination “c:\CakeWizard\tools\perforce_sync.bat” -GuestUser “administrator” -GuestPassword ‘pass1234′

#execute the script we’ve copied

Invoke-VMScript -ScriptText “c:\CakeWizard\tools\sources_sync.bat” -VM $vm -GuestUser ‘administrator’ -GuestPassword ‘pass1234′

The ScriptText parameter can point to another script or be the script itself. So that makes it pretty easy to be included in Hudson/Jenkins job.

The script you execute returns live results to your screen while working inside the VM. This is great for monitoring the progress of a long test run or analyzing test run results. The best thing about the Invoke-VMScript cmdlet is that it can execute many types of scripts: Powershell, BAT for Windows and Bash for Linux systems!

Now that we’ve prepared everything, it’s good time to create another snapshot. Let’s name it “Ready for test execution” or “Clean state”.

[Install product]

Let’s upgrade to the latest build version of our product. We have a Powershell script that does the job. Let’s see how to execute it:

Invoke-VMScript -ScriptText “c:\CakeWizard\Tools\upgradeProduct.ps1″ -VM $vm -GuestUser ‘administrator’ -GuestPassword ‘pass1234′

[Create report]

You can generate a test run execution report after the test execution. Our report holds information about the test id, test status, log files and execution times.

Invoke-VMScript -ScriptText “c:\CakeWizard\Tools\generateReport.bat” -VM $vm -GuestUser ‘administrator’ -GuestPassword ‘pass1234′ 

# locally copy a report

Copy-VMGuestFile -GuestToLocal -VM $vm -Source “c:\Tests\Results\Report\*.*” -Destination “d:\TestResults\Report\*.*” -GuestUser “administrator” -GuestPassword ‘pass1234′

[Clean up environment / Revert to snapshot]

When you run a test (again, you can use Invoke-VMScript to trigger execution), you might pollute your brand new Test_Environment_VM_1. Wouldn’t it be great, if you could always go back to a clean state of your test environment? This way you can be comfortable with running even the deadliest, most insane, build-crashing tests. Our new script for creating a new test environment could do the trick, but there’s an even easier way to fix the ruined VM – snapshots!

# reverting the vm to a previous state

Set-VM  –VM $vm –Snapshot “Clean state”

“Set up once – test multiple versions!” 

You can easily customize these one-line scripts for your own use cases. By combining all of the scripts, you will automate your virtual environment! You are just a step away from provisioning multiple test environments!

[Copy VM to create a new virtual test environment]

A VM is just a set of files, so we can copy a VM by copying its files. Let’s create a new environment called “Test_Environment_VM_2”.

#change directory to ‘vmstores:’. It gives you access to the datastore tree.

#navigate to your datastore and Test_Environment_VM_1 location and copy it

cd vmstores: 

Copy-Item “.\Test_Environment_VM_1” -Destination “Test_Environment_VM_2” –Recurse

#now let’s register the new VM

New-VM –VMFilePath “[datastore_name] .\Test_Environment_VM_2\Test_Environment_VM_2.vmx” –Name “Test_Environment_VM_2”| Start-VM

And you are ready with your new environment and John can help you with testing “Cake Wizard”

Note – there is an easier way to clone a virtual machine – check VM templates, a feature available in vCenter Server.

Virtualization makes your testing process quite agile and scalable – you can create more VMs on which to run tests, than the number of the physical machines you can buy or power up. Even your developer colleagues can use the VMs for developing the software in the first place!

You can also easily revert a VM to a clean state, without losing any time for configuration.

You can set up and configure the VM so that most of the things that you can do with a physical machine are possible (except for testing specific/special hardware devices).

You will no longer fear of hardware failures, because your VM can be automatically migrated with no downtime.

You can automate all these tasks with the highly-rated tool – VMware vSphere PowerCLI.

Well, Michael, I think that now you understand the advantages of virtualization in testing. As quirky as this concept sounds, you’ll find that the basics are not that different from the classical approach. Do you have any other questions, we can discuss over a cup of coffee?