/****************************************************************************
**
** Copyright (C) 2016 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the Qt Creator documentation.
**
** Commercial License Usage
** Licensees holding valid commercial Qt licenses may use this file in
** accordance with the commercial license agreement provided with the
** Software or, alternatively, in accordance with the terms contained in
** a written agreement between you and The Qt Company. For licensing terms
** and conditions see https://www.qt.io/terms-conditions. For further
** information use the contact form at https://www.qt.io/contact-us.
**
** GNU Free Documentation License Usage
** Alternatively, this file may be used under the terms of the GNU Free
** Documentation License version 1.3 as published by the Free Software
** Foundation and appearing in the file included in the packaging of
** this file. Please review the following information to ensure
** the GNU Free Documentation License version 1.3 requirements
** will be met: https://www.gnu.org/licenses/fdl-1.3.html.
**
****************************************************************************/

/*!
    \contentspage {Qt Creator Manual}
    \previouspage creator-cpu-usage-analyzer.html
    \page creator-autotest.html
    \nextpage creator-advanced.html

    \title Running Autotests

    \QC integrates the \l{Qt Test} framework and
    \l{https://github.com/google/googletest}{Google C++ Testing Framework} for
    unit testing applications and libraries. You can use \QC to build and run
    Qt tests, Qt Quick tests (QML-based Qt tests), and Google tests for your
    projects.

    \image qtcreator-autotests.png

    To enable the experimental Auto Test plugin, select \uicontrol Help >
    \uicontrol {About Plugins} > \uicontrol Utilities > \uicontrol {Auto Test}.
    Restart \QC to be able to use the plugin.

    \section1 Creating Qt Tests

    You can use a wizard to create projects that contain Qt tests:

    \list 1
        \li Select \uicontrol File > \uicontrol {New File or Project} >
            \uicontrol {Other Project} > \uicontrol {Auto Test} >
            \uicontrol Choose to create a project with boilerplate code for a
            Qt test.

        \li In the \uicontrol {Project and Test Information} dialog, specify
            settings for the project and test:

            \list 1

                \li Select the \uicontrol {GUI Application} check box to create
                    a Qt application.

                \li In the \uicontrol {Test case name} field, enter a name for
                    the test case.

                \li Select the \uicontrol {Requires QApplication} check box to
                    add the include statement for QApplication to the main.cpp
                    file of the project.

                \li Select the \uicontrol {Generate initialization and cleanup
                    code} checkbox to add functions to your test that are
                    executed by the testing framework to initialize and clean
                    up the test.

                \li In the \uicontrol {Build auto tests} field, select
                    \uicontrol Always to always build the test when building
                    the project or \uicontrol {Debug Only} to only build it
                    during debug builds.

            \endlist

    \endlist

    \QC creates the test in the \c{tests\auto} directory in the project
    directory. Edit the .cpp file to add private slots for each test
    function in your test. For more information about creating Qt tests, see
    \l{Creating a Test}.

    \section1 Setting Up the Google C++ Testing Framework

    To build and run Google tests, you must have the Google C++ Testing
    framework installed and configured on the development host. You can either
    clone it from Git Hub or install it from an installation package.

    To configure a project to use a cloned Google testing framework, edit the
    \c INCLUDEPATH variable in the project file (.pro) to include the source
    and \c include folders of Google Test's \c googletest and \c googlemock.
    Usually, you need to add the following subfolders:

    \list
        \li \c googletest
        \li \c googlemock
        \li \c googletest/include
        \li \c googlemock/include
    \endlist

    You also need to add the necessary files to the \c SOURCES variable. For
    example:

    \list
        \li \c googletest/src/gtest-all.cc
        \li \c googlemock/src/gmock-all.cc
    \endlist

    To configure a project to use an installed Google testing framework package,
    add the following include paths to the .pro file:

    \list
        \li \c <googletest_install_path>/include/gtest
        \li \c <googletest_install_path>/include/gmock
    \endlist

    Then add linker options to be able to find the libraries and to link against
    them. For example, for qmake based projects, you typically need to add the
    following values to the .pro file:

    \list
        \li \c {LIBS += -lgtest -L<path_to_gtest_lib>}
        \li \c {LIBS += -lgmock -L<path_to_gmock_lib>}
    \endlist

    To specify settings for running Google tests, select \uicontrol Tools >
    \uicontrol Options > \uicontrol {Test Settings}.

    \image qtcreator-autotests-options.png

    To run disabled tests, select the \uicontrol {Run disabled tests} check box.

    To run several iterations of the tests, select the \uicontrol {Repeat tests}
    check box and enter the number of times the tests should be run in the
    \uicontrol Iterations field. To make sure that the tests are independent and
    repeatable, you can run them in a different order each time by selecting the
    \uicontrol {Shuffle tests} check box.

    For more information about creating Google tests, see the
    \l{https://github.com/google/googletest/blob/master/googletest/docs/Primer.md}
    {Google Test Primer}.

    \section1 Building and Running Tests

    To build and run tests:

    \list 1

        \li Open a project that contains tests.

        \li In the \uicontrol {Test Results} output pane, select
            \inlineimage run_small.png
            (\uicontrol {Run All Tests}) to run all test or
            \inlineimage qtcreator-run-selected-tests.png
            (\uicontrol {Run Selected Tests}) to run the selected tests.

            You can select the tests to run in the \uicontrol Tests view in the
            \uicontrol Projects pane. For more information, see
            \l{Viewing Tests}.

            \note By default, \QC builds a project before deploying and running
            it.

    \endlist

    If a test takes more than a minute to execute, the default timeout might
    stop the test execution. To increase the timeout, select \uicontrol Tools >
    \uicontrol Options > \uicontrol {Test Settings}.

    \QC scans the project for tests when you open the project and updates the
    test list when you edit tests. To only update the test list when the
    \uicontrol Tests view or the \uicontrol {Test Results} output pane is open,
    deselect the \uicontrol {Always parse current project for tests} check box.

    The code inside a benchmark test is measured, and possibly also repeated
    several times in order to get an accurate measurement. This depends on the
    measurement back-end that you can select in \uicontrol {Test Settings}:
    walltime, CPU tick counter, event counter, Vallgrind Callgring, and Linux
    Perf. For more information, see \l{Creating a Benchmark}.

    \section1 Viewing Test Output

    The test results are displayed in the \uicontrol {Test Results} output pane.

    \table
        \header
            \li Result
            \li Description
        \row
            \li BENCH
            \li Benchmark test.
        \row
            \li BFAIL
            \li Blacklisted test case failed. Since Qt 5.4, you can
                provide a BLACKLIST file for tests. It is mainly used internally
                by the Qt CI system.
        \row
            \li BPASS
            \li Blacklisted test case passed.
        \row
            \li DEBUG
            \li Debug message.
        \row
            \li XFAIL
            \li Test case is expected to fail, so it is marked by using the
                QEXPECT_FAIL macro. If the test case passes instead, an
                unexpected pass (XPASS) is written to the test log.
        \row
            \li FAIL
            \li Test case failed. Double-click the line for more information.
        \row
            \li INTERNAL
            \li Internal message.
        \row
            \li PASS
            \li Test case passed.
        \row
            \li SKIP
            \li Test case was skipped.
        \row
            \li XPASS
            \li Test case passed even though it was expected to fail.
        \row
            \li WARN
            \li Warning message.
    \endtable

    To view only messages of a particular type, select
    \inlineimage filtericon.png
    (\uicontrol {Filter Test Results}), and then select the types of messages to
    show.

    To hide internal messages and run configuration warnings, select
    \uicontrol Tools > \uicontrol Options > \uicontrol {Test Settings}.
*/
