test kitchen always success because wrong test directory name
All causes are that I change suite name.
kitchen verify suites name and test directory name
I change suites name.
kitchen test
finish successfully.
I put new test case to fail.
but kitchen test still successfully finished.
I execute
kitchen converge
and
kitchen verify
result
-----> Starting Kitchen (v1.4.1)
-----> Setting up <ucsc-genomebrowserbbbb-ubuntu-1404>...
Finished setting up <ucsc-genomebrowserbbbb-ubuntu-1404> (0m0.00s).
-----> Verifying <ucsc-genomebrowserbbbb-ubuntu-1404>...
Preparing files for transfer
Transferring files to <ucsc-genomebrowserbbbb-ubuntu-1404>
Finished verifying <ucsc-genomebrowserbbbb-ubuntu-1404> (0m0.00s).
-----> Kitchen is finished. (0m1.09s)
It seems to finish successful.
$ echo $?
0
Why I changed the suites name ?
According to my memory, when I executed test kitchen, virtualbox name looks like same. And I set aside those environment to check what happens. I did not destroy immediately Result in I could not recognized virtualbox name and environment.
Changed CI style
After that, I change two things
- Always destroy kitchen environment. Jenkins remember the test result. And I rerun from scratch.
- Change suites name
But latter one is not good for me.
I change suites name to default again
So I rewrite suites name to default. Everything goes finished
Another solution Change test directory name same as suites name
Another solution, renamed test directory name from default to new suites name. verify process successfully executed.
Check result output after verify process.
I want to see and certainly if verify process had been executed.
Using Publish JUnit test result plugin on Jenkins.
This plugin creates graph of JUnit results.
If there is no test file, this plugin aborted.
Concept to check result output
- delete result files
- exec kitchen test
Clean before checkout git plugin
This plugin offer various options.
I add Clean before checkout behavior.
After setup this, always clean the directory.
Check truly clean
I use free style job.
Before I execute kitchen test.
I check there is no xml file under the test result directory. If there is xml file. Job is fail.
To avoid reuse previous test result.
Using rspec_junit_formatter to output JUnit xml file.
To output result as XML(JUnit), I use
get serverspec output result file from virtualbox
Sync virtual machine and host machine using synced_folder.
Serverspec2 and resource
serverspec-init
Following command creates sample files. It’s useful for people who has some old knowledge.
serverspec-init