/*!
\page script_unit_testing Unit/System Testing Using Scripts

Creating of Unit/System tests based on scripts follows the rules defined on the
\ref script_testing.

To achieve this, two separate files will have to be created:

@li The test script which will contain the test code.
@li The validation file which will contain the validations.

\section auto_load_scripts Automatic Script Loading
The Shell test suite automatically identifies and executes the test scripts
existing at \a \<shell_src_root\>/unittest/scripts/auto

Inside of that folder, the next sub-folder will be scanned for test scripts:

For JavaScript:
@li js_adminapi: should contain tests for the Admin API
@li js_devapi: should contain tests for the X Dev API
@li js_shell: should contain tests for the Shell API

For Python:
@li py_adminapi: should contain tests for the Admin API
@li py_devapi: should contain tests for the X Dev API
@li py_shell: should contain tests for the Shell API

These folders must contain two sub-folders:
@li scripts: which contain the scripts with the test code
@li validation: which contains the validation files

For every script added to the \a scripts folder, a validation file with the
exact same name should exist on the \a validation folder.

\subsection auto_environment Unit Test Environment

The constructor of the
<a class ="el" href="classtests_1_1_shell__test__env.html">Shell_test_env</a>
class describes the environment variables required to setup a testing
environment for the shell.

The attributes on that class are set based on the configured environment, and a
similar set of variables is available when the test scripts are executed.

The list of variables available out of the box on test scripts are:

@li \a __user: The user as defined in \a MYSQL_URI.folders
@li \a __system_user: Login name of the user who start process. Currently
logged in user on system.
@li \a __mysql_uri: A password-less URI for MySQL protocol sessions.
@li \a __mysql_uripwd: A full URI for MySQL protocol sessions (password included).
@li \a __xhost_port: \a __host + ":" + \a _port
@li \a __host_port: \a __host + ":" + \a _mysql_port
@li \a __schema: 'mysql'
@li \a __mysql_sandbox_port1: Port for the first sandbox.
@li \a __mysql_sandbox_port2: Port for the second sandbox.
@li \a __mysql_sandbox_port3: Port for the third sandbox.
@li \a __sandbox_uri1 = URI for the first sandbox.
@li \a __sandbox_uri2 = URI for the second sandbox.
@li \a __sandbox_uri3 = URI for the third sandbox.
@li \a __sandbox_uri4 = URI for the fourth sandbox.
@li \a __sandbox_uri5 = URI for the fifth sandbox.
@li \a __sandbox_uri6 = URI for the sixth sandbox.
@li \a localhost: 'localhost'";
@li \a hostname_ip: TBD
@li \a testutil: An instance of the
<a class ="el" href="classtests_1_1_testutils.html">Testutils</a> class.
@li \a __version: The version of the MySQL Server being used for the tests.

From the variables listed above, \a testutil deserves special mention as it is
a testing module with lots of utility functions that make it easier to create
tests.

\subsection creating_tests Creating New Tests
Creating a new test file is as simle as creating the test script divided in
chunks as described in <a class ="el" href="script_testing.html">The Script
Based Testing Framework</a>.

To do this, you can simply use the shell and start trowing instructions there
and copy/paste the ones that make sense on your script file, choose wisely how
to create the chunks, in general, each chunk should be a very simple test.

Once you are happy with the script, copy it into the one of the folders
described at \ref auto_load_scripts.

Now identify the filter required to execute that test, use:
\code
run_unit_tests --gtest_list_tests
\endcode

This will list all the tests available, you can use a starting filter to
identify the ones of your interest:

@li For JS AdminAPI Tests use: --gtest_filter=Admin_api_scripted/Auto_script_js*
@li For JS DevAPI Tests use: --gtest_filter=Dev_api_scripted/Auto_script_js*
@li For JS Shell Tests use: --gtest_filter=Shell_scripted/Auto_script_js*
@li For PY AdminAPI Tests use: --gtest_filter=Admin_api_scripted/Auto_script_py*
@li For PY DevAPI Tests use: --gtest_filter=Dev_api_scripted/Auto_script_py*
@li For PY Shell Tests use: --gtest_filter=Shell_scripted/Auto_script_py*

For example to list the Admin API tests, execute:

\code
run_unit_tests --gtest_list_tests --gtest_filter=Admin_api_scripted/Auto_script_js*

At the end you will get the listing as follows:

Admin_api_scripted/Auto_script_js.
  run_and_check/0  # GetParam() = "js_adminapi/bug26159339_adminapi_does_not_take_group_name_into_account_interactive.js"
  run_and_check/1  # GetParam() = "js_adminapi/clear_read_only.js"
  run_and_check/2  # GetParam() = "js_adminapi/mismatched_group_name.js"
  run_and_check/3  # GetParam() = "js_adminapi/reboot_cluster_super_read_only_interactive.js"
\endcode

There you can identify the \a run_and_check/# that belongs to the script
you just created, use it as follows to execute your test:

\code
run_unit_tests --gtest_filter=Admin_api_scripted/Auto_script_js.run_and_check/1
\endcode

\subsubsection automatic_validations Automatic Validation File Generation

If you tried executing your brand new test, most probably you got a errors like:

\code
MISSING VALIDATIONS FOR CHUNK
\endcode

This is because you have not created the validation file. This process can be
done by hand, as explained in <a class ="el" href="script_testing.html">The Script
Based Testing Framework</a>, but there is an easier way.

Append \a --generate-validation-file to the command you used to execute your
test, as follows:

\code
run_unit_tests --gtest_filter=Admin_api_scripted/Auto_script_js.run_and_check/1 --generate-validation-file

You will see something like this:

[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Admin_api_scripted/Auto_script_js
[ RUN      ] Admin_api_scripted/Auto_script_js.run_and_check/1
Test script: js_adminapi/clear_read_only.js
[       OK ] Admin_api_scripted/Auto_script_js.run_and_check/1 (524 ms)
[----------] 1 test from Admin_api_scripted/Auto_script_js (524 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (524 ms total)
[  PASSED  ] 1 test.
\endcode

This mode of execution will cause every chunk on your script file to be executed
and then for each chunk, it will grab all the content of \a STDOUT and \a STDERR
and create the validation file.

\b IMPORTANT: Most probably a lot of the information on this validation file
is valid only for your environment, but if it is used on a different environment
most probably the validations will fail.

To fix this, you must edit the validation rules and properly set single line
validations where it applies, also replace all the data that depends on your
environment like hostnames, ports, connection strings and so on with dynamic
variables available on the test environment so they automatically take the right
values before the validation occurs.

\b NOTE: the automatically generated validation file will have the same name as
the script, but on the validation folder. If you execute the recording mode
again a new file will be generated on the same location but appending the .new
extension to it, if you execute the recording once again, the previous .new file
will be overwritten.

\subsection good_practices Testing Good Practices
The next recommendations are considered good practices while creating new tests.

@li Reuse existing tests: it is very common that existing tests already setup
the context required to verify enhancements, bug fixes or new features, instead
of creating a whole new test for such scenarios, reuse the existing test code
to reduce the amount of work required, as well as to keep the test suite in a
maintainable state.

@li Design integral tests: When a new feature is added, rather than creating
a bunch of tests that execute the same setup steps once and again and again
and again, create a bigger test that executes the initial setup once and then
use code to evolve the test to get new conditions for additional checks, that's
why the chunk logic has been created.

@li File names for tests should identify what the content is about,for example
collection_create_index.js would contain tests that are related to the
Collection.createIndex function (and all its variants). A good practice is to
use filenames in the format as \a \<api\>_\<object\>.js or
\a \<api\>_\<api_object\>_\<function\>. for Unit Tests, for system tests which
are more complex the next format is suggested \a system_\<description\>.js where
\<description\> describes what the test is about.

@li Create self contained tests: every single test must be designed in a way
that can be executed isolated from the rest of the test suite.
*/
