Showing posts with label code. Show all posts
Showing posts with label code. Show all posts

Tuesday, September 18, 2012

Powershell - the good and the bad. The story of a survivor.

Today's topic is Powershell - it's usefulness and weirdness.

It is the Microsoft super cool console which is used to provide command-line API for all of MS big products like Office, SQL Serveer, etc. It is build on top of .Net so every .Net class you write, you can take advantage of it in the command-line.

It has it's advantages:
- using objects and not strings for pipelining commands
- the .net framework
- ease-of-use - with just a few commands you can open up the Powershell world.
- very readable
- good help about_xxx articles

Let's start with the
Good Stuff

Pipelining is a powerful feature. You can pass multiple result objects from 1 command to the next one just like this:

How you delete everything inside a folder ?
Get-Item "c:\folder\" | Remove-Item -Force -Recurse #alias is dir or ls
How to see how to use a command ? What are its parameters? What does it do?
Get-Help "Get-Item" -Full  #alias is man or help 
What are the properties/methods of a process ?
Get-Process | Select-Object -First 1 | Get-Member #alias is gm

With just Get-Help (or man), Get-Member (or gm) and Get-Command (to list all available commands/functions/aliases) , you can do anything!


You'll use pipelining all the time! It's really cool!

Some nice hits are:
- filtering a collection of items by a property. Returns all folders with name ending with the string "project"
$folderList | ? { $_.Name -match ".*project" }
which is short way to write:
Foreach-Object $folder in $folderList | Where-Object {  $folder.Name -match ".*project"}

- doing a repetitive task on many objects. Display

$folderList = dir | ? {$_.GetType().Name -match ".*DirectoryInfo.*" }
$folderList | % { Write-Host "$($_.Name) - $($_.CreationTime)" }
- errors
When there is an error just get the automatic variable $Error (array of error objects) and check the last 5 errors $Error[0..5]


Main usage of Powershell is doing some quick check on something in the Microsoft infrastructure or writing a script to automate a routine tedious task like creating a report or killing all not-responding processes at the moment.
Overall system administrators will love it.
...

and here ends the good things... If you try to use it for something else, you'll hit many different obstacles:

Strange behaviour

Functions want to return something!

In fact, every function you call that returns an object, and the result is not saved in a variable, is automatically transfered as a result. OK, but when you have multiple such calls inside your function, these results are added into an array and here's the /unexpected behaviour/...
function myFunc(){
 Get-Process
 dir
}
$b = myFunc
$b.GetType()  --- #it says array

The other way to stop this from happening is to enclose the call with brackets and [void] the result like this:
 [void] (Get-Process)
The strange empty array

When comparing empty array to $null  with the comparison operator -eq (equals), doesn't return neither $true, nor $false...just ... nothing.

This is partially explained with the Powershell ability to auomatically flatten an array to its elements. When it's an empty array - the operation doesn't execute. But there's no error as well... or warning.

Try this:
@() -eq $null
and
$null -eq @()
Also if you are used to separate each function parameter with comma, you will have a hard time!

Using comma ( ,) will generate an array. Well, Powershell likes arrays a lot, and you will, too, but only after the initial struggle.


Using it for plain programming

Unfortunately, when you want to use it to code features, and even small frameworks, you hit

some obstacles like
- Visual Studio doesn't support it... at all! No other IDE with satisfying experience, too
- PowerShell ISE isn't feature-rich enough
- only suitable is Notepad++ with word completion and many macros along the way...
- no compile-time catching bugs like - unused variables, or undefined ones...


Unfinished or just plain broken features


Modules as Custom objects

As of Powershell v2, there is a new method to package commands (cmdlets) - modules (instead of snapins).

So you can import modules into the runspace or import them as CustomObject, which has
methods as the functions declared in the .psm1 file (module script file)

These CustomObjectModules are very buggy when it comes to multiple imports and generating exceptings...Trust me. The latter is very nasty...
I
magine writing a few modules which should use each other - common utils, etc. And then you have an exception. Now where exactly did the exception occur ? Try to get catch it with try-catch - the Error record contains only the row/execution line of your current module in development! Nothing about the inner exception or error stack... !! Which is not useful at all.

In order to be productive with these modules, I ended up writing my own ExceptionStackTrace building function which collected errors, line numbers, positioned the stack trace correctly, and stops when it reaches another stack trace (.Net one) which is descriptive enough...  Well, I'm sure it could have been done better, but it works stable enough and I like that it solved this problem which should have been handled by the Powershell Team!

Also nested imports fail from time to time with functions which cannot be found...
I am seriously, not going to use Modules as CustomObjects anymore... Too bad. Almost like Object-oriented programming... but with serious issues...

Remoting
While Powershell provides nice API for remote calls - Invoke-Command for executing single scriptblock and Enter-PSSession for interactive mode. They are thought quite nice, but unfortunately, the last few weeks I've come to the conclusion that they are quite fragile and I was often the observer of the following results:
- OutOfMemory Error
- Stopped execution without errors
- WinRM was not configured correctly errors pointing to articles in MSDN which weren't specific enough.


So basically here are my findings about Powershell Remoting:

- complex requirements for remote machine for the remoting to work. Not a single full guide for this.

- adding snapins remotely for machine with no Internet access - 2 minutes ! You have to turn off the Internet Options > Advanced > Security option to "Check for publisher's certificate invocation" and it will pass in seconds.

- remoting stops without any error or logs. Hard to debug. Could be missing configurations/rights, could be low memory

- slightly more complex remoting produces many errors. For example starting a new resource-hungry process on the remote machine results in OutOfMemory Error. Solved by increasing the memory per PowerShell property described at the end of the article.

- inconsistent behaviour - different errors for unchanged code

- lack of documentation for some of the features - only a few blog posts saved me a few hours/days

A few tips here:

- the remoting script you run should be as light as possible and should just execute a file/script located on the remote machine. This was you can lower remoting data transfers(logs, commands, variables, etc)

- for complex, time-consuming tasks use Invoke-Command xxxx -AsJob parameter. This way it will create invisible background thread, which will take even less resources. Actually, for my complex scenario , it proved to be the only working option...

Get-Item wsman:localhost\Shell\MaxMemoryPerShellMB 
Set-Item wsman:localhost\Shell\MaxMemoryPerShellMB 512 -Force

Conclusion:

Don't get me wrong. Using Powershell is overall an easy and straight-forward experience, but when you try something a bit more kinky and ... let's say that the 'fun' starts there.

Best regards,
Powershell-ill
Leni Kirilov

Tuesday, October 19, 2010

DevReach 18+19 October Sofia, Bulgaria (part5)

...continued

Designing Applications in the era of Many-core Computing:

This session was about making a unparalleled program use all cores of a CPU.
The presenter was a Romanian so his program was the recipee for making goulash.
He used threading on a higher level with a structure called a Task.
A tasks can be combined to be one after another by a lambda expressions.


Building RESTful Applications with the Open Data Protocol

The last session I attended was one regarding RESTful services.
The presenter was very high-spirited and it was a joy to listen to his explanations.
So REST services is exposing a service to be accessed  via pure URL address through the browser. 
Using the HTTP methods POST GET UPDATE DELETE to access the service if you have the access rights.

He demonstrated how to connect to such a service via a client.

He created an REST service. 
He showed an easy API to do paging for the service.

It was a very nice presentation. 

L.K. 

Thursday, September 09, 2010

Java 7 new feautres (part 2)

Hi there,

I'm continuing my review on the expected features from where I started them:

JSR 308: Annotations on Java types
An extension to the Java annotation syntax to permit annotations on any occurrence of a type
----
In my opinion, this will make Java code stranger... May be I have got used to the current Java code style, but too much @ will make it look like PHP or other languages using strange symbols.
About the feature - it could add new flexibility in checks - kinda like "throws", final ... Imagine how JUnit will change after this feature is introduced!


JSR TBD: Small language enhancements (Project Coin)
A set of small language changes intended to simplify common, day-to-day programming tasks
----
String in switch --- My comment is "FINALLY, GUYS!"
Automatic resource management --- Well, we all got tired of all the try-catch around code pieces using  Streams. Just makes the code larger, but not doing anything special.
Generics initialization improvement - personally, I don't mind leaving it out, but sloppy developers will make less errors.
Collection literals and array-like access - this is similar to the one above. Just for easier initialization and access.



JSR 203: More new I/O APIs for the Java platform (NIO.2)
New APIs for filesystem access, scalable asynchronous I/O operations, socket-channel binding and configuration, and multicast datagrams
---
The best new feature about this is the new .resolve() method, which will find the files as they are meant to be found and not force the developer to think which class loader is used at the moment etc etc... Until man encounters this problem, he doesn't know how frustrating it can be.



Update the XML stack
Upgrade the JAXP, JAXB, and JAX-WS APIs to the most recent stable versions
Dropped
---
Actually, I don't know the exact reason behind the decision to drop the upgrade of the above-mentioned technologies, but as I use them at work on daily basis, I find this surprising. It would be great if their specifications are revised and developers at Oracle (former Sun) upgrade them to the next level, because Java API for XML will always be "modern" and the need for updates/improvements won't decrease.
Of course, I could be terribly wrong. ;)

Swing JDatePicker component
Add the SwingLabs JXDatePicker component to the platform
---
I have not used Swing as much as to notice that DatePicker is missing ... When I read it was to be "ADDED" I thought, "shame on you for forgetting it"! I suppose that more than half of Swing applications need date of some sort...

-------------------

I'll skip intentionally the other improvements regarding class loaders, collections concurrency and Swing , because I don't find them as attractive as the ones I commented above. Here's a link to the full list.

I hope you found my comments reasonable. I am optimistic about the new Java version. I just hope the developers don't have too many problems and they don't delay the launch...

Good luck!



Leni Kirilov

Thursday, September 02, 2010

My own naming convention and what problems could arise from it

Hi, developers!

We write applications and our applications use data. We create data, edit data, store data...
Sometimes we store data on the File System. And I think that's a common practice.

Have you ever wondered how to name your files in order, not to have naming collisions or creating some temporary files with some extensions, only you as a developer, should know about ?
Have you ever encountered exceptions in your code while accessing the file system? A glitch in your code maybe?

I am going to address this issue because it's sometimes regarded as a trivial task and everybody is said to do it the right way.

1. Create states in your program 

If at runtime your program has some strict set of states that it is in, then you can be calm and happy about your design. If you can keep the state of your variables consistent, you can manipulate them as you wish and not fear some collisions. What I mean is when you invoke a method, it knows exactly what to expect - whether a String that is passed as a parameter could be null, or if it is in a special internal format, etc...

Entering methods and exiting from methods must keep the representation invariant intact and everybody should know their role in the process.

2. Keep the number of states of your program low
If you apply rule no.1, you can still mess up, if you have like ... 100 different states. Your head will be full of conditions and rules to transit from one state to the other...

 A developer should not think about more than 7 things while developing. That way he can stay focused and make always the best decisions about his design.

3. Document your states

Be honest about your program's states. Tell your colleagues. Don't be shy or overprotective , if there's this colleague who always critisize your code. If they know about the states you introduced, then they could tell you about a problem earlier and adopt those states in their modules early on.
Not to mention that at one happy moment, somebody else different from you will be supporting this code.

4. Avoid tricky (smart) code

I've seen smart code and it lead to invisible problems, reported and fixed years after the code was written and got into the final product.

- Imagine the following situation - you are passed String with the name of a file, and you must create your own temp file, where you should store your temporary data during the processing of the original file. So you create a file with

String tempName = originalFile + "temp";

And later you decide to wipe all those temp files, but since you have added "temp" at the end, you can just use
if ( "temp".equals(String.lastIndexOf(aFileName) ) {...deleteFile(); }

And you thought you deleted your files. What happens if there are files that are called "xxxtemp" by the original user? Even if you add "temp" at the end , you will get to delete both the temp and the original.

- Another smelly situation - your use case is to keep the temp files and delete all non-temp-ending files.
Ok so lets use:

fileName.replaceAll("", "temp");

So we'll discard the temp and we'll automatically get the original filename...
Wrong! What happens if all this is happening in directory "c:\temp\myfiles" ? You will get invalid file paths and another exception is thrown.


You should be very careful with these kind of situations with files.
I suggest 2 ways to avoid problems:

A) store in a collection the original files and in a new collection the references to the temp (new) files.
That way you don't need to think of weird algorithms to find temp/original files by their derivative's names.

B) if you have to think of such an algorithm be sure that it finds exactly what you want.
Back to the example with the replaceAll():
int lastIndex = fileName.lastIndexOf("temp");
fileName.replace("", "temp", lastIndex);

And this is exactly the opposite of string += otherString.


I am sure other developers have found other techniques to avoid problems but those were mine.
Feel free to share your opinion.


Leni Kirilov

Sunday, July 11, 2010

Idea: JUnit and Code coverage connection

I'm going to share my first thoughts regarding unit testing, JUnit and TDD (test-driven development). I'm sure that this won't be the last post on this subject because there is a lot that can be said and I've had a few interesting discussions with a colleagues of mine regarding the benefit of TDD and whether it is a good or a bad principle.

---

Here I'm going to share a thought which seems logic to me, but unfortunately have not seen it implemented yet.
I've been writing some unit tests recently and  it occurred to me that it would be nice if the JUnit was combined with code coverage and after runs of the test suites, there is a statistic that shows which test (test method), which lines of code did they cover!

Imagine if it was possible to see that - when an old test fails, it shows you which particular code it validates, so if a change in that code occurred, you can catch it immediately!

Of course there is the scenario when a change occurred in the context of the process that is tested and not the process itself and the code that could break a certain test, but nevertheless this information won't lie to you and trick you and slow you down in rooting down the real problem.

Also some monitoring of the submits to the project repository and analyzing which code fixes what test , would be the same as which test validates which lines of code.

There could be done something about that!


----
Some basic tips:

For NetBeans users check this link:
http://netbeans.org/kb/docs/java/junit-intro.html

Check official xUnit site or Junit
http://www.junit.org/

L.K. 

Sunday, June 20, 2010

NetBeans plugin Code Coverage "interesting" behaviour

I decided to test the plugin for code coverage for NetBeans. Created an empty test method, activated the plugin, ran the tests and when I ask for coverage statistics it said I had not ran my tests...

I lost an hour looking for the cause of this problem but at the end all was pretty innocent...

Once I implemented some actual methods and not juts an EMPTY test method , and at least 1 line of code was invoked the statistics appeared and everything was fancy and flashy and showing some numbers...

So remember - it's not the problem in your plugin! It works... (except for single test files ) but when you run all tests, it works as nice as a baby soft skin :)

L.K.

NetBeans 6.8 ME removal leads to unusable NetBeans

I had installed NetBeans 6.8 the version where all possible java technologies are bundled together.

I had no problems, but I decided to deactivate and uninstall the ME plugins... huge mistake.. The whole NetBeans became an empty shell... with just a few menus to the top and a simple editor... I was greatly surprised.

The good thing is that I installed NetBeans 6.9 and it sniffed my NetBeans 6.8 and imported all projects, copied the configurations and even the plugins... And in just 5 minutes a could continue with my work having all my configurations, no need to configure all my custom changes all over again.

So in just 30 minutes I was surprised twice by the NetBeans - the good thing is that in the end everything is working how it is supposed to be working. :)

I suppose this is something that Еclipse lacks...

//later edit:
- different Eclipse installations can share configuration with the Workspaces

Sunday, June 06, 2010

Programming language popularity

I've just stumbled upon a very interesting site showing the trend through the years regarding peoples' interest in different programming languages.

You can find it here:

http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

TIOBE says their statistics don't show that the languages compared are better/worse than the others, or that there are more Lines Of Codes written using them now.

Wednesday, May 19, 2010

NetBeans 6.8 bugs

Today I'll comment my favourite IDE for Java development - NetBeans.

Some may say that I'm a newbie and like less complicated IDE or that "true Java developers" use Eclipse - OK so be it, but I'm working on Eclipse at work and at home I'm using whatever I feel like using and NetBeans is what I like using!

So I've recently found a few bugs while developing a small application.

1. Swing forms editor bug

NetBeans is popular with its very well-made Swing UI editor and I agree with common understandings. UI forms are easy to use, can be configured in lots of ways (some of which I haven't tried yet), but satisfies my needs completely. It generates some code and locks it , so that future UI replacement/movements are still consistent with the rest of the class you are managing. Easy event handling methods and so on...

But I've had a few places I wanted to manipulate and configure the initialization of the components. NetBeans automatically creates a constructor with "init();" method call in it , but my case required to modify this init() method.

So you can click on the JComponent and click "Customize code" , which lets you modify the constructor called but not the declaration! I found this very restrictive and tried to trick NetBeans.

I had to change the place of a class from package "a" to package "b", but if I cannot change the declaration, I'm stuck. So I decided to use REFACTOR rename on the package of the problematic class. And it worked! ...
at first...

I refactored the name of the package and no problem in the UI created code.
But if you try to move 1 pixel of the UI form , it regenerates again the locked code and refreshes the metainf. xml in the netbeans project and my IDE shows an error of non-existing class/package...

I'm going to report this and get it fixed in NetBeans 6.9.

2. Main class in built .jar is missing

The next major bug I've found is already reported. I even reached the helpful guys in www.stackoverflow.com
You can find my question here ->  Main class in NetBeans bug

I'll rewrite the short description here:

You create a Java project in NetBeans 6.8 or 6.9 and build it to create a .jar file.
Even if you have set in the project settings which class contains the main() method to be executed, the build omits this configuration in the meta-inf/manifest file  after build and so the .jar is not executable.

This is easily fixed by :
- creating a metainf.mf file in root of your project
- the file contains something like

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.1
Created-By: 16.3-b01 (Sun Microsystems Inc.)
Main-Class: com.example.MainClass
Class-Path: lib/lib1.jar lib/lib2.jar


- editing the nbproject/project.properties  file
- add manifest.file=manifest.mf
- save and rebuild

This solved my problem and thanks a lot to the NetBeans and stackoverflow society so quickly.

L.K.

Wednesday, May 12, 2010

Debugging Eclipse

Today I had the chance to debug some functionality in the Eclipse IDE, or to be more precise the SAP NetWeaver Developer Studio customized version of Eclipse.

The main thing about debugging is to setup your environment the right way.
There is configuration.ini file in the main directory of your IDE. Inserting specific parameters allows the NetWeaver DS to be started in debug mode and on specified port.
Then you need to start a debug session as a "Remote application" on 'localhost:port' and you are good to go.

 I managed to do that but I lost some time figuring this thing out :
- better make a copy of your IDE and not debug your IDE with the same one.

What I mean is that you can open 2 times the same IDE and debug one of them with the other but here's the tricky part. If you want to make update to a file in the IDE you want to debug, you will have to shut down both instances and start them up again. And lets suppose you have to change not 1 file but a few files, and not just 1 time, but a few times... and then you have wasted your time.


Personal advice - make a separate copy of your IDE and use it for DEBUG and leave your main IDE untouched.

 

Other than that, there is nothing special about it.

L.K.

Just a reminder for my future posts

I'll write about the following topics in no particular order, unless somebody says explicitly that they are interested in a topic and I'll post it ASAP.

I'd like to remind myself to document my thoughts regarding these topics:

These topics will follow:
  • JDK 7 new features
    • what I like
    • what I don't like 
  • JDK 1.6 installation problems
    • hints how to solve the problems 
  • Sony PRS-600 Touch - my brand new reader (and all the fuss regarding getting it)
    • review
    • some hints, tips and tricks 
    • problems 
    • conversion tools 
  • Windows 7
    • connecting to network
    •  control panel (Back) button navigation
  • some bugs of my Dell Inspiron Laptop
    • DVD
    • strange noise 
  • Eclipse debugging
  • NetBeans 6.8 bugs
    • swing editor bugs - needs to be reported
    • main class problem
  • Laptop battery preservation  tips
    • because what I have done so far has proven me right - thoughts from experience
  • Internet Browsers
    • Opera thoughts
    • Chrome
    • Firefox
    • IE
  • Google Chrome OS
  • Android
  • Skype time stamp bug 
  • HDD partitioning problems
    • how not to make your data inconsistent because of pure mistake or lack of knowledge
  •  Test-Driven Development

    L.K.

Hello world!

Hello world!

This blog is going to be especially for people interested in computers, programming, Java and mobile gadgets.

I've decided to start this blog as a way to express some of my thoughts which I forget otherwise.

I hope everybody who encounters this blog, finds something interesting and valuable.

Happy reading!

L.K.