天道酬勤,学无止境

'cmake rebuild_cache' for *just* a subdirectory?

I have an issue with the generation of makefiles stage of CMake being slow which is similar to this unanswered question:

CMake is slow to generate makefiles

My project is made up of a top level CMakeLists.txt file which uses add_subdirectory() to add various subprojects for individual library and executable components.

For a given component, the CMakeLists.txt file contains something like:

add_library(mylib SHARED
  sourceFile1.cpp
  sourceFile2.cpp
  ...
)

I can build just the contents of that directory using:

make mylib

If I modify the CMakeLists.txt file in the sub-directory (which I've been doing a lot as part of a migration from pure Makefiles to CMake) then run make it correctly re-runs CMake to update the configuration as if I'd run make rebuild_cache.

However, I notice that it in fact reconfigures the entire project. I really want for CMake to be clever enough to know it only needs to regenerate the Makefile in the current directory and sub-directories.

Is there a better way to structure a CMake project to achieve this? I see some people use project() for each CMakeLists.txt in each sub-project. In general, is this a good idea?

Alternatively/additionally is there some way to speed up the generation step of CMake? (currently I have 60s+)

Bonus points if you want to discuss why CMake itself should or shouldn't be able to run in parallel (imagine a cmake -j).


I've added the meson-build tag as a modest bounty, but alone it hasn't yet attracted enough attention to warrant an answer. It's this kind of problem that might cause people to switch to build systems to meson-build (assuming it doesn't have similar problems) or something similar.

It is possible that the correct answer is it can't be done without modifying the source to CMake. To earn the bounty though I require an explanation in terms of how CMake works and/or where it is flawed.


Clarification: It is the generation step that is slow in my case. The configure itself is quick enough, but CMake hangs for a quite a while between outputting "-- Configuring done" and "-- Generating done".

For a full cache rebuild I run:

make -n rebuild_cache
Running CMake to regenerate build system... 
using Makefile generator
-- FOOBAR_VERSION: 01.02.03
-- Configuring done
-- Generating done
-- Build files have been written to: /home/brucea/work/depot/emma/main/cmake
real 74.87
user 1.74
sys 1.02

Under the hood this runs:

cmake -H<source dir> -B<build dir>

I presume -B is a synonym for --build. Neither option is described correctly in the documentation. -H is the root of the source directory (not the same as --help as the documentation would have you believe).

It's fast to get to the output of "Configuring done", but slow from there:

For example,

15:44:14 execve("/usr/local/bin/cmake",
>grep Generating cmake_strace.log
>grep "Configuring" cmake_strace.log
15:44:15 write(1, "-- Configuring done\n", 20-- Configuring done
15:45:01 write(1, "-- Generating done\n", 19-- Generating done
>grep "Build files" cmake_strace.log
15:45:22 write(1, "-- Build files have been written"..., 77-- Build files have been written to:

If editing a single CMakeLists.txt file in a subdirectory, and then running make -n, it runs:

cd /home/project/cmake && /usr/local/bin/cmake -H/home/project/cmake -B/home/project/cmake --check-build-system CMakeFiles/Makefile.cmake 0

--check-build-system is another undocumented option.

The effect is the same - regenerate the whole build system, not just the current subtree. There is no difference in behaviour between an in-source and an out-of-source build.

If I run a trace, e.g.:

strace -r cmake --trace -H/home/project/cmake -B/home/project/cmake 2>&1 | tee cmake_rebuild_cache.log
sort -r cmake_rebuild_cache.log | uniq

The majority of time spent seems to be spent on (or between) open, access & unlink calls.

The length of each task is quite variable, but the huge number of them builds up. I have no idea what the Labels.json and Labels.txt files are about (something internal to CMake).

One run:

    49.363537 open("/home/projectbar/main/test/foo2bar/CMakeFiles/test2.foo2bar.testViewingSource1.dir/build.make", O_RDONLY) = 5
     1.324777 access("/home/projectbar/main/test/performance/CMakeFiles/performancetest.chvcthulhu.testChvcthulhuPerformance2.dir", R_OK) = 0
     0.907807 access("/home/projectbar/main/test/foo2bar/CMakeFiles/test2.foo2bar.testPeripheralConnection2.dir", R_OK) = 0
     0.670272 unlink("/home/projectbar/main/src/foo2bar/Foo2Bar/CMakeFiles/foo2bar_lib.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.600272 access("/home/projectbar/main/test/foo2bar/testFilesModel2.ok", R_OK) = 0
     0.599010 access("/home/projectbar/main/test/hve2snafu/testInvalidByte2c.ok", R_OK) = 0
     0.582466 read(5, "openjdk version \"1.8.0_71\"\nOpenJ"..., 1024) = 130
     0.570540 writev(3, [{"# CMAKE generated file: DO NOT E"..., 8190}, {"M", 1}], 2) = 8191
     0.553576 close(4)                  = 0
     0.448811 unlink("/home/projectbar/main/test/snafu2hve/CMakeFiles/test2.snafu2hve.testNoProbes2.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.431559 access("/home/projectbar/main/src/foo2bar/Foo2Bar/CMakeFiles/foo2bar_lib.dir", R_OK) = 0
     0.408003 unlink("/home/projectbar/main/test/lachesis/CMakeFiles/test2.lachesis.testBadSequenceNumber1.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.407120 write(4, "# The set of languages for which"..., 566) = 566
     0.406674 write(3, "# CMAKE generated file: DO NOT E"..., 675) = 675
     0.383892 read(3, "ewingPeriod.cpp.o -c /home/bruce"..., 8191) = 8191
     0.358490 unlink("/home/projectbar/main/cmake/CMakeFiles/mklinks.chvdiff.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)

Another run of the same command:

     2.009451 unlink("/home/projectbar/main/cmake/CMakeFiles/mklinks.lachesis.dir/Labels.json") = -1 ENOENT (No such file or directory)
) = 20
) = 19
     1.300387 access("/home/projectbar/main/test/chvedit/CMakeFiles/test2.chvedit.tefooultiMatchFactoringEdit2.dir", R_OK) = 0
     1.067957 access("/home/projectbar/main/test/chvedit/CMakeFiles/test2.chvedit.tefooultiMatchFactoringEdit2.dir", R_OK) = 0
)         = 1
     0.885854 unlink("/home/projectbar/main/src/gorkyorks2bar/CMakeFiles/doxygen.correct.gorkyorks2bar.dir/Labels.json") = -1 ENOENT (No such file or directory)
     0.854539 access("/home/projectbar/main/test/reportImpressions/ReportImpressions/CMakeFiles/testsuite1_reportImpressions.dir", R_OK) = 0
     0.791741 unlink("/home/projectbar/main/cmake/CMakeFiles/mklinks.bar_models.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.659506 unlink("/home/projectbar/main/cmake/CMakeFiles/mklinks.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.647838 unlink("/home/projectbar/main/test/libyar/YarModels/CMakeFiles/testsuite1_yarmodels.dir/Labels.txt") = -1 ENOENT (No such file or directory)
     0.620511 unlink("/home/projectbar/main/test/libyar/YarModels/CMakeFiles/testsuite1_yarmodels.dir/Labels.json") = -1 ENOENT (No such file or directory)
     0.601942 unlink("/home/projectbar/main/cmake/CMakeFiles/mklinks.lachesis.dir/Labels.txt") = -1 ENOENT (No such file or directory)
     0.591871 access("/home/projectbar/main/src/runbardemo/simple_demo/CMakeFiles", R_OK) = 0
     0.582448 write(3, "CMAKE_PROGRESS_1 = \n\n", 21) = 21
     0.536947 write(3, "CMAKE_PROGRESS_1 = \n\n", 21) = 21
     0.499758 unlink("/home/projectbar/main/test/foo2bar/CMakeFiles/test2.foo2bar.testInputDirectory1.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.458120 unlink("/home/projectbar/main/test/yak2dcs/CMakeFiles/test2.yak2dcs.testsuite2.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.448104 unlink("/home/projectbar/main/test/reportImpressions/CMakeFiles/test2.reportImpressions.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.444344 access("/home/projectbar/main/src/bananas/CMakeFiles/bin.bananas.dir", R_OK) = 0
     0.442685 unlink("/home/projectbar/main/test/rvedit/CMakeFiles/test2.rvedit.tefooissingOptionValue.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.425604 unlink("/home/projectbar/main/test/listdcs/CMakeFiles/test2.listdcs.testListCalls5.dir/progress.make.tmp") = -1 ENOENT (No such file or directory)
     0.391163 access("/home/projectbar/main/src/siedit/CMakeFiles/siedit.dir", R_OK) = 0
     0.362171 access("/home/projectbar/main/test/foo2bar/CMakeFiles/test2.foo2emma.testHowResults6.dir", R_OK) = 0

Note the Ninja generator is much faster (though still not brilliant). For example,

/usr/bin/time -p ninja rebuild_cache
ninja: warning: multiple rules generate ../src/ams2yar/ams2yar. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
ninja: warning: multiple rules generate ../src/vox/vox. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
ninja: warning: multiple rules generate ../src/bananas/bananas. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
ninja: warning: multiple rules generate ../src/fidlertypes2fidlerinfo/fidlertypes2fidlerinfo. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
ninja: warning: multiple rules generate ../src/mkrundir/mkrundir. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
ninja: warning: multiple rules generate ../src/runyar/runyar. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
ninja: warning: multiple rules generate ../src/runyardemo/runyardemo. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
[1/1] Running CMake to regenerate build system...
Generator=Ninja
-- FOO_VERSION: 01.02.03
-- Configuring done
-- Generating done
-- Build files have been written to: /home/project/cmake/build
real 12.67
user 1.01
sys 0.31

Note that the project is not quite ready for Ninja yet as there are errors like:

ninja: warning: multiple rules generate ../src/runfoobardemo/runfoobardemo. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]

and

ninja: error: dependency cycle: ../src/foobar -> ../src/foobar/CMakeFiles/foobar -> ../src/ams2emma/foobar

to be resolved. This question is really about why the Makefile generator is slow. I'm not sure if the problems Ninja shows are useful hints here or red herrings.


Building CMake with more optimisations does not help.

Based on my trace output it and the output of time, it is unlikely that it would. The user time and therefore time spend within the CMake code itself is quite low. (see e.g. What do 'real', 'user' and 'sys' mean in the output of time(1)?).

Here's what I tried for completeness:

export CXX_FLAGS="-O3 -ftree-vectorise -msse2"
cmake -DCMAKE_BUILD_TYPE=RELEASE

Actually using a more optimised CMake does make the configure part faster, but in my case it is the generate part that is slow. It would appear from the timing that this step is somehow I/O bound.


I decided to investigate Florian's idea that using memory insteam of file stream for temporary files might make a difference.

I decided to try the easy route and hacked CMake to write .tmp files to a RAM disk instead.

I then went the whole hog and tried generating the build system on the RAM disk:

sudo mkdir /mnt/ramdisk
sudo mount -t tmpfs -o size=512m tmpfs /mnt/ramdisk
/usr/bin/time -p cmake -H/<source> -B/mnt/ramdisk/build

I was very surprised to find this makes no difference at all to the wall clock time:

real 59.61
user 1.55
sys 0.62
>du -sh /mnt/ramdisk/build/
4.4M    /mnt/ramdisk/build/

Similarly with ramfs:

real 51.09
user 1.58
sys 0.50

What could be happening here? I was guessing sub-processes, but I can't work out which sub-processes are consuming the wall clock time if any of them are. They look to be very short lived.


For completeness, here is some output from perf (CMake built with -fno-omit-frame-pointer):

perf record -g --call-graph dwarf cmake -H<source> -B<build>
perf report -g graph
Samples: 17K of event 'cycles', Event count (approx.): 14363392067
  Children      Self  Command          Shared Object                 Symbol
+   65.23%     0.00%  cmake            cmake                         [.] do_cmake
+   65.02%     0.00%  cmake            cmake                         [.] cmake::Run
+   60.32%     0.00%  cmake            cmake                         [.] main
+   59.82%     0.00%  cmake            libc-2.17.so                  [.] __libc_start_main
+   57.78%     0.00%  cmake            cmake                         [.] _start
+   55.04%     0.00%  cmake            cmake                         [.] cmGlobalUnixMakefileGenerator3::Generate
+   54.56%     0.00%  cmake            cmake                         [.] cmake::Generate
+   49.90%     0.00%  cmake            cmake                         [.] cmGlobalGenerator::Generate
+   38.87%     0.02%  cmake            cmake                         [.] cmLocalUnixMakefileGenerator3::Generate
+   18.65%     0.01%  cmake            cmake                         [.] cmMakefileTargetGenerator::WriteTargetBuildRules
+   17.05%     0.02%  cmake            cmake                         [.] cmMakefile::ExecuteCommand
+   16.99%     0.01%  cmake            cmake                         [.] cmMakefile::ReadListFile
+   16.84%     0.01%  cmake            cmake                         [.] cmCommand::InvokeInitialPass
+   16.79%     0.00%  cmake            cmake                         [.] cmMakefile::Configure
+   14.71%     0.00%  cmake            cmake                         [.] cmMakefile::ConfigureSubDirectory
+   14.67%     0.05%  cmake            cmake                         [.] cmMacroHelperCommand::InvokeInitialPass
+   14.27%     0.02%  cmake            cmake                         [.] cmMakefileUtilityTargetGenerator::WriteRuleFiles
+   13.91%     0.00%  cmake            cmake                         [.] cmGlobalGenerator::Configure
+   13.50%     0.05%  cmake            cmake                         [.] cmOutputConverter::Convert
+   13.48%     0.00%  cmake            cmake                         [.] cmAddSubDirectoryCommand::InitialPass
+   13.46%     0.00%  cmake            cmake                         [.] cmMakefile::AddSubDirectory
+   12.91%     0.00%  cmake            cmake                         [.] cmGlobalUnixMakefileGenerator3::Configure
+   12.82%     0.00%  cmake            cmake                         [.] cmake::ActualConfigure
+   10.90%     0.00%  cmake            cmake                         [.] cmake::Configure
+   10.55%     0.02%  cmake            cmake                         [.] cmMakefileTargetGenerator::WriteObjectRuleFiles
+   10.35%     0.09%  cmake            cmake                         [.] cmLocalUnixMakefileGenerator3::WriteMakeRule
+    9.76%     0.03%  cmake            cmake                         [.] cmMakefileTargetGenerator::WriteObjectBuildFile
+    7.97%     0.00%  cmake            cmake                         [.] cmMakefileLibraryTargetGenerator::WriteRuleFiles
+    7.93%     0.00%  cmake            cmake                         [.] cmMakefileExecutableTargetGenerator::WriteRuleFiles
+    7.88%     0.00%  cmake            cmake                         [.] cmLocalUnixMakefileGenerator3::WriteLocalMakefile
+    7.68%     0.02%  cmake            [kernel.kallsyms]             [k] sysret_audit
+    7.60%     0.05%  cmake            [kernel.kallsyms]             [k] __audit_syscall_exit
+    7.40%     0.08%  cmake            cmake                         [.] cmsys::SystemTools::CollapseFullPath

And perf report -g graph -no-children:

+    2.86%  cmake            libc-2.17.so                  [.] _int_malloc
+    2.15%  cmake            libc-2.17.so                  [.] __memcpy_ssse3_back
+    2.11%  cmake            [kernel.kallsyms]             [k] find_next_bit
+    1.84%  cmake            libc-2.17.so                  [.] __memcmp_sse4_1
+    1.83%  cmake            libc-2.17.so                  [.] _int_free
+    1.71%  cmake            libstdc++.so.6.0.20           [.] std::__ostream_insert >
+    1.18%  cmake            libstdc++.so.6.0.20           [.] std::basic_string, std::allocator >::~basic_string
+    1.13%  cmake            libc-2.17.so                  [.] malloc
+    1.12%  cmake            cmake                         [.] cmOutputConverter::Shell__ArgumentNeedsQuotes
+    1.11%  cmake            libstdc++.so.6.0.20           [.] std::string::compare
+    1.08%  cmake            libc-2.17.so                  [.] __strlen_sse2_pminub
+    1.05%  cmake            cmake                         [.] std::string::_S_construct
+    1.04%  cmake            cmake                         [.] cmsys::SystemTools::ConvertToUnixSlashes
+    0.97%  cmake            cmake                         [.] yy_get_previous_state
+    0.87%  cmake            cmake                         [.] cmOutputConverter::Shell__GetArgument
+    0.76%  cmake            libstdc++.so.6.0.20           [.] std::basic_filebuf >::xsputn
+    0.75%  cmake            libstdc++.so.6.0.20           [.] std::string::size
+    0.75%  cmake            cmake                         [.] cmOutputConverter::Shell__SkipMakeVariables
+    0.74%  cmake            cmake                         [.] cmOutputConverter::Shell__CharNeedsQuotesOnUnix
+    0.73%  cmake            [kernel.kallsyms]             [k] mls_sid_to_context
+    0.72%  cmake            libstdc++.so.6.0.20           [.] std::basic_string, std::allocator >::basic_string
+    0.71%  cmake            cmake                         [.] cmOutputConverter::Shell__GetArgumentSize
+    0.65%  cmake            libc-2.17.so                  [.] malloc_consolidate
+    0.65%  cmake            [kernel.kallsyms]             [k] mls_compute_context_len
+    0.65%  cmake            cmake                         [.] cmOutputConverter::Shell__CharNeedsQuotes
+    0.64%  cmake            cmake                         [.] cmSourceFileLocation::Matches
+    0.58%  cmake            cmake                         [.] cmMakefile::ExpandVariablesInStringNew
+    0.57%  cmake            cmake                         [.] std::__deque_buf_size
+    0.56%  cmake            cmake                         [.] cmCommandArgument_yylex
+    0.55%  cmake            cmake                         [.] std::vector >::size
+    0.54%  cmake            cmake                         [.] cmsys::SystemTools::SplitPath
+    0.51%  cmake            libstdc++.so.6.0.20           [.] std::basic_streambuf >::xsputn

评论

There are so many aspects that define CMake's configuration and generation steps duration (besides what you actually do in your CMakeLists.txt files; it's e.g. your host system, your toolchain and which CMake version/distribution you are using).

So I try to concentrate on the specific questions you have.

Rebuild/rewrite makefiles for just a subdirectory?

For the start: Using add_subdirectory() is good for structuring your CMake code. But you have to keep in mind that you can always change global CMake properties in a subdirectory and that targets inside those subdirectories can have cross-dependencies.

So what does CMake do (considering the "I have touched one CMakeLists.txt file in a subdirectory" case discussed here):

  • If a CMakeLists.txt file is changed it goes through the complete hierarchy of CMakeLists.txt files again and rebuilds the build environment again in memory.
  • Now it temporarily recreates all the necessary build/make files and checks if they defer from the existing ones (see cmGeneratedFileStreamBase::Close()).
  • If a file has changed it replaces the existing one with the new one.

This behavior is necessary because any makefile can change even when only a subdirectory's CMakeLists.txt file has changed and it was optimized to prevent unnecessary rebuilds during the actual make step (from touched makefiles).

Is there some way to speed up the generation step of CMake?

So yes, it does temporarily rewrite all makefiles (which could be slow) and no, you can't minimize this while using add_subdirectory() to only the changed subdirectory.

Maybe one possible performance optimization for the future in CMake's own code would be to use memorystreams instead of filestreams for the temporary files.

@BruceAdams tested this by using a RAM disk for the generated makefile environment with no effect.

And yes, the CMake generated cmake_check_build_system rule does almost the same as the rebuild_cache rule and yes the used -B, -H and --check-build-system options are CMake internal command line options and therefore undocumented (even if often referred to on Stack Overflow, e.g. in one of my answers here).

What helped me to speed-up the configuration/generation was to rebuild CMake itself with a lot more optimization options than the normal distributions and using a 64-bit toolchain instead of the 32-bit versions currently still distributed.

Here are some test results (using the CMake test script found below with 100 subdirectories/libraries) on my Windows PC always using the same MSYS environment, but different CMake compilations of the same CMake source code:

  1. Official CMake 3.2.2 version:

    $ time -p cmake -G "MSYS Makefiles" ..
    [...]
    real 43.93
    user 0.00
    sys 0.03
    
  2. Using mingw32 and GNU 4.8.1 I rebuild CMake 3.2.2 with

    cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-O3" -G "MSYS Makefiles" ..
    

    and got

    $ time -p /c/temp/cmake-3.2.2/MSYS32/bin/cmake.exe -G "MSYS Makefiles" ..
    [...]
    real 41.37
    user 0.01
    sys 0.04
    

    And the same with my antivirus software turned off:

    $ time -p /c/temp/cmake-3.2.2/MSYS32/bin/cmake.exe -G "MSYS Makefiles" ..
    [...]
    real 20.98
    user 0.00
    sys 0.04
    
  3. Using mingw-w64 and GNU 5.3.0 I rebuild CMake 3.2.2 with

    $ cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-march=native -m64 -Ofast  -flto" -G "MSYS Makefiles" ..
    

    and got

    $ time -p /c/temp/cmake-3.2.2/MSYS64/bin/cmake.exe -G "MSYS Makefiles" ..
    [...]
    real 25.59
    user 0.00
    sys 0.04
    

    And the same with my antivirus software turned off:

    $ time -p /c/temp/cmake-3.2.2/MSYS64/bin/cmake.exe -G "MSYS Makefiles" ..
    [...]
    real 6.95
    user 0.00
    sys 0.03
    

To summarize I see two main influences:

1st: The configuration step can be speed-up by going for a 64-bit version and optimize for your processor platform (you would certainly have to find a common base -march=... or -mtune=... for all your project's build PCs).

2nd: The generation step can mostly be sped up by searching for possible file I/O bottlenecks outside of CMake. In my case telling the antivirus software to not check the toolchain and build directories each time I read/write to those was really speeding up things.

Remark: I confirmed @BruceAdams test results that the the compiler's auto-vectorization (default for -O3 or -Ofast) is not able to do much about CMake source code's ability to run in multiple processes/on multiple cores.

Is there a better way to structure a CMake project to achieving this?

Yes, if you e.g. know that a certain sub-tree of your CMake script code just generates a library and has no dependencies, you could put that part in an external project by using ExternalProject_Add(). And yes, having had similar concerns regarding large CMake projects, this is seen as a good "modern CMake" practice (see also references below).

References

  • CMake Performance Tips
  • CMake: Parallel generation of make files
  • GCC: how is march different from mtune?
  • CMake: How to set up source, library and CMakeLists.txt dependencies?
  • Making CMake library accessible by other CMake packages automatically
  • Use CMake-enabled libraries in your CMake project

What I used to reproduce your problem

Just for completeness and if someone wants to check those numbers against his/her own, here is my test code:

cmake_minimum_required(VERSION 3.0)

project(CMakeTest CXX)

#set_property(GLOBAL PROPERTY GLOBAL_DEPENDS_DEBUG_MODE 1)

set(_idx 1)

while (_idx LESS 100)
    math(EXPR _next_idx "${_idx} + 1")
    if (NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/lib${_idx}")
        file(MAKE_DIRECTORY "lib${_idx}")
        file(
            WRITE "lib${_idx}/lib${_idx}.h"
                "int lib${_idx}_func();"
        )
        file(
            WRITE "lib${_idx}/lib${_idx}.cc"
                "#include \"lib${_next_idx}.h\"\n"
                "int lib${_idx}_func() { return lib${_next_idx}_func(); }"
        )
        file(
            WRITE "lib${_idx}/CMakeLists.txt"
                "add_library(lib${_idx} \"lib${_idx}.cc\")\n"
                "target_link_libraries(lib${_idx} lib${_next_idx})\n"
                "target_include_directories(lib${_idx} PUBLIC \".\")"
        )
    endif()
    add_subdirectory("lib${_idx}")
    set(_idx "${_next_idx}")
endwhile()

if (NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/lib${_idx}")
    file(MAKE_DIRECTORY "lib${_idx}")
    file(
        WRITE "lib${_idx}/lib${_idx}.h"
            "int lib${_idx}_func();"
    )
    file(
        WRITE "lib${_idx}/lib${_idx}.cc"
            "int lib${_idx}_func() { return 0; }"
    )
    file(
        WRITE "lib${_idx}/CMakeLists.txt"
            "add_library(lib${_idx} \"lib${_idx}.cc\")\n"
            "target_include_directories(lib${_idx} PUBLIC \".\")"
    )
endif()
add_subdirectory("lib${_idx}")

if (NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/main.cc")
    file(
        WRITE "main.cc"
            "#include \"lib1.h\"\n"
            "int main() { return lib1_func(); }"
    )
endif()

add_executable(${PROJECT_NAME} "main.cc")
target_link_libraries(${PROJECT_NAME} lib1)

And then - after the first cmake .. and make calls - doing:

$ touch ../lib100/CMakeLists.txt
$ time -p cmake ..
-- Configuring done
-- Generating done
-- Build files have been written to: [your path here]
real 28.89
user 0.01
sys 0.04

受限制的 HTML

  • 允许的HTML标签:<a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • 自动断行和分段。
  • 网页和电子邮件地址自动转换为链接。

相关推荐
  • How to speed up Compile Time of my CMake enabled C++ Project?
    I came across several SO questions regarding specific aspects of improving the turn-around time of CMake enabled C++ projects lately (like "At what level should I distribute my build process?" or "cmake rebuild_cache for just a subdirectory?"), I was wondering if there is a more general guidance utilizing the specific possibilities CMake offers. If there is probably no cross-platform compile time optimization, I'm mainly interested in Visual Studio or GNU toochain based approaches. And I'm already aware of and investing into the generally recommended areas to speed up C++ builds: Change
  • 如何加快启用CMake的C ++项目的编译时间?(How to speed up Compile Time of my CMake enabled C++ Project?)
    问题 我最近遇到了几个有关改善CMake支持的C ++项目的周转时间的特定方面的问题(例如“我应该在什么级别分发构建过程?”或“仅对子目录使用cmake rebuild_cache?”),我当时是想知道是否有利用CMake提供的特定可能性的更一般的指导。 如果可能没有跨平台的编译时间优化,那么我主要对基于Visual Studio或GNU toochain的方法感兴趣。 而且我已经意识到并投资于一般推荐的领域,以加快C ++的构建: 更改/优化/微调工具链优化您的代码库/软件体系结构(例如,减少依赖关系并使用定义明确的子项目-单元测试) 购买更好的硬件(SSD,CPU,内存) 就像这里,这里或这里推荐的一样。 因此,我在这个问题上的重点是第一点。 另外,我知道可以在CMake Wiki中找到的建议: CMake:以您的所有核心进行构建 CMake性能提示 前者只处理基本知识(并行生成),而后者则主要处理如何加快CMake文件的解析速度。 为了使这一点更加具体,如果我从这里使用100个使用MSYS / GNU的库的CMake示例,我得到以下time测量结果: $ cmake --version cmake version 3.5.2 CMake suite maintained and supported by Kitware (kitware.com/cmake). $ time
  • Add dependency to the CMake-generated build-system itself
    In short: I know how to add dependencies to targets, in a CMake-generated build system. But I would like to add dependencies to the generated build-system itself. Longer question: In the CMake-generated build process of cgal, we would like CMake to automatically re-run the configuration step, when certain files are modified. Unneeded details are hidden below: As a matter of fact, we generate using CMake the build system for the CGAL libraries/examples/demos, but also at the same time the build system for our Doxygen-generated documentation. The Doxyfile is generated from multiple files. When
  • CPack:从子目录(googletest目录)中排除INSTALL命令(CPack: Exclude INSTALL commands from subdirectory (googletest directory))
    问题 我将CMake用于一个项目,并将googletest用于我的测试用例。 环顾互联网,将googletest源代码复制到存储库的子文件夹中并将其包含在“ add_subdirectory(googletest)”中似乎是一种常见的做法。 我做到了 现在,我正在使用CPack为我的项目生成debian软件包。 不幸的是,由CPack生成的软件包与我的项目一起安装了googletest。 这当然不是我想要的。 在googletest目录中查找时,我在此处找到了一些INSTALL cmake命令,因此很清楚为什么会发生这种情况。 现在的问题是-我该如何避免呢? 我不喜欢从googletest修改CMakeLists.txt文件,因为我必须记住在更新时重新应用我的修改。 还有另一种方法可以在CPack中禁用这些安装吗? 回答1 更新:如另一个答案中所述,似乎EXCLUDE_FROM_ALL选项是在子目录的子项目中禁用install的最直接,最正确的方法: add_subdirectory(googletest EXCLUDE_FROM_ALL) 先前的解决方案 如果您不需要在项目版本中进行测试(您想随CPack一起交付),请有条件地包含googletest子目录,并在打包时将有条件的条件设置为false: ... if(NOT DISABLE_TESTS) add
  • CMake: How to set up source, library and CMakeLists.txt dependencies?
    I have several projects (all building with CMake from the same source tree structure) all using their own mix out of dozens of supporting libraries. So I came about the question how to set up this correctly in CMake. So far I have only found CMake how to correctly create dependencies between targets, but I'm still struggling between setting up everything with global dependencies (the project level does know it all) or with local dependencies (each sub-level target only handles its own dependencies). Here is a reduced example of my directory structure and what I currently came up with using
  • 具有多个可执行文件的CMake共享库(CMake share library with multiple executables)
    问题 我的项目包含共享一些通用代码的几个可执行文件。 我想将公共代码放在可执行程序可以链接到的静态库中。 (通用代码很小,我不想处理共享库)。 源代码树如下所示: 项目 CMakeLists.txt 常见的 CMakeLists.txt src 包括 app1 src CMakeLists.txt app2 src CMakeLists.txt app1和app2都取决于共同的代码。 此通用代码是非常特定于应用程序的,因此此目录树之外的其他项目将永远不需要使用该通用代码。 因此,我不希望不在任何全局位置安装该库。 顶级CMakeLists.txt文件仅添加了子目录: project(toplevel) cmake_minimum_required(VERSION 3.1) add_subdirectory(common) add_subdirectory(app1) add_subdirectory(app2) 公用库的CMakeLists.txt文件创建静态库并设置包含目录: add_library(common STATIC common.cpp) target_include_directories(common PUBLIC "${CMAKE_CURRENT_LIST_DIR}/include") 可执行文件如下所示: project(app1) cmake_minimum
  • cmake, lost in the concept of global variables (and PARENT_SCOPE or add_subdirectory alternatives)
    I have a cmake project in which I have some modules and I'm using Find-*.cmake for including the shared modules in the application. For not taking in account every module that I add, I have defined a kind of global LIB variables tor the linker: # inside a Find-*.cmake or in the CMakeLists.txt of the modules: set(LIB ${LIB} ...) so in one of the final applications that uses some modules I can just do: target_link_libraries(${APP_NAME} ${LIB}) Then, I'd like to have the compiled modules in the /project_path/modules/foo/build so that if a module is really big to compile it can be compiled once
  • CMake: How do I change properties on subdirectory project targets?
    I'm trying to organize the targets in my subproject (in this case poco), but I've come to find that properties can't be modified for ALIAS targets. I want the targets in my external project to be in their own folder, instead of sprawled out everywhere in the project tree (say the visual studio generator). Is there an easier way to add projects with my own properties? So instead of: - CMakePredefinedTargets - ALL_BUILD - INSTALL - ... - MyTargets - SomeLibrary - SomeExe - CppUnit - Crypto - Data - ... I want: - CMakePredefinedTargets - ALL_BUILD - INSTALL - ... - MyTargets - SomeLibrary -
  • CMake share library with multiple executables
    My project contains several executables that share some common code. I would like to put the common code in a static library that the executables can link to. (The common code is pretty small and I prefer not to deal with shared libraries). The source tree looks something like this: project CMakeLists.txt common CMakeLists.txt src include app1 src CMakeLists.txt app2 src CMakeLists.txt app1 and app2 both depend on the code in common. This common code is very application specific and will never need to be used by another project outside this directory tree. For that reason I would prefer not to
  • cmake does not find qt 5.1.1
    I just installed the Qt 5.1.1 for Windows 32-bit (MinGW 4.8, OpenGL) and tried to add it to my cmake. But CMake just does not want to find it and i dont know why. What am i missing? Do i need to set an environment variable or something? Here is my cmake : cmake_minimum_required( VERSION 2.8.11 ) PROJECT(Blemmer) # Find includes in corresponding build directories set(CMAKE_INCLUDE_CURRENT_DIR ON) # Instruct CMake to run moc automatically when needed. set(CMAKE_AUTOMOC ON) # Detect and add SFML set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake/Modules" ${CMAKE_MODULE_PATH}) set( CMAKE_RUNTIME
  • Keeping file hierarchy across subdirectories in CMake
    Till date I still do not really understand what the 'best practice' is for doing this for a CMake project with many subdirectories. Say I have a project hierarchy as such and each subdirectory has source files in it... --CMake Project Source dir |-- SubD1 |-- SubSubD1 |-- SubD2 What I would usually do is to do add_subdirectory(SubD1) and respectively for D2 in the CMakeLists.txt of the root directory and recursively for the subdirectory in the CMakeLists.txt of the SubD1 directory, while declaring variables in each subdirectory and making them visible in the root directory with PARENT_SCOPE
  • How is CMake used? [closed]
    Closed. This question needs to be more focused. It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by editing this post. Closed 3 years ago. Improve this question It is notoriously difficult to get any useful information on CMake as a beginner. So far, I've seen a few tutorials on how to set up some very basic project or another. However, none of these explain the reasoning behind anything that is shown in them, always leaving many holes to fill. What does calling CMake on a CMakeLists mean? Is it supposed to be called
  • 立即构建工具,以便以后可以在同一CMake运行中使用(Building a tool immediately so it can be used later in same CMake run)
    问题 我有一个有趣的鸡肉和鸡蛋问题,并有一个潜在的解决方案(请参阅我发布的答案),但是该解决方案以不寻常的方式使用CMake。 更好的替代方案或评论将受到欢迎。 问题: 问题的简单版本可以描述为具有以下特征的单个CMake项目: 构建目标之一是命令行可执行文件,我将其称为mycomp ,其源代码位于mycompdir中,并且无法对该目录的内容进行任何修改。 该项目包含文本文件(我将它们foo.my和bar.my ),这些文件需要在它们上运行mycomp才能生成一组C ++源代码和头文件以及一些CMakeLists.txt文件,这些文件定义了从这些源代码构建的库。 同一项目中的其他构建目标需要链接到那些生成的CMakeLists.txt文件定义的库。 这些其他目标还具有#include某些生成的标头的源。 您可以将mycomp视为类似于编译器,并将步骤2中的文本文件视为某种源文件。 这带来了一个问题,因为CMake在配置时需要CMakeLists.txt文件,但是mycomp在构建时才可用,因此在首次运行时无法尽早创建CMakeLists.txt文件。 无应答: 通常,基于ExternalProject的超级构建安排可能是解决此问题的方法,但以上内容是我正在处理的实际项目的相当大的简化,并且我没有自由将构建划分为不同的部分或执行其他大型任务规模重组工作。 回答1 问题的症结在于
  • Handling header files dependencies with cmake
    I am using CMake on a small C++ project and so far it works great... with one twist :x When I change a header file, it typically requires recompiling a number of sources files (those which include it, directly or indirectly), however it seems that cmake only detects some of the source files to be recompiled, leading to a corrupted state. I can work around this by wiping out the project and rebuilding from scratch, but this circumvents the goal of using a make utility: only recompiling what is needed. Therefore, I suppose I am doing something wrong. My project is very simply organized: a top
  • CMake: In which order are files parsed (cache, toolchain, etc.)?
    This seems as a trivial question, since CMake is a script language the general answer is: strictly sequential. But I came across several cases where it was important when or in which order CMake is parsing certain files. So I wonder: Is there a documentation available that describes the order in which files (incl. internal CMake files) are parsed? Is the file order depending on the CMake version or some CMake options/settings/environment incl. the chosen generator or host environment? The cases I came across so far, where the above information was important: The toolchain file is parsed before
  • No tests found when using gtest with cmake/ctest
    I have a project with the following structure: linalg ├── build ├── CMakeLists.txt ├── docs │ └── Doxyfile ├── include │ └── linalg │ └── vector3.hpp ├── src │ ├── CMakeLists.txt │ └── linalg │ └── vector3.cpp └── test ├── CMakeLists.txt └── linalg └── test_vector3.cpp The file test_vector3.cpp is a gtest unit test file which provides two simple tests. The top level CMakeLists.txt simply sets up the includes and adds the src and test subdirectories: cmake_minimum_required(VERSION 2.8) project(linalg) include_directories(include) add_subdirectory(src) add_subdirectory(test) The src/CMakeLists
  • 有关在CMake / CDash中使用gcov的详细指南?(Detailed guide on using gcov with CMake/CDash?)
    问题 我在项目中使用CMake,并设置了cdash服务器以进行连续/每晚构建。 一切正常,通过设置crontab,我们每小时/每晚的构建/测试结果会自动上传到我们的cdash服务器。 我的下一步是将测试覆盖率报告添加到构建中。 我在http://www.cmake.org/Wiki/CTest:Coverage上找到该文档,但是坦率地说,它与实用指南有点不同。 目前,我已经添加了必需的标志(而不是-fprofile-arcs -ftest-coverage ,我发现--coverage更好),编译过程将生成.gcno文件。 但是后来我被困住了。 命令 make NightlyCoverage 似乎什么也没做。 有人可以告诉我下一步该怎么做吗? 我想要的结果是通过make NightlyCoverage ,生成覆盖率报告并将其上传到cdash服务器。 回答1 我一直在成功使用https://github.com/bilke/cmake-modules/blob/master/CodeCoverage.cmake。 只需遵循准则:将文件添加到我的CMAKE_MODULE_PATH目录中,添加 set(CMAKE_MODULE_PATH ${PROJECT_SOURCE_DIR}/CMakeModules) if(CMAKE_COMPILER_IS_GNUCXX) include
  • CMake: Project structure with unit tests
    I am trying to structure my project to include the production sources (in src subfolder) and tests (in test subfolder). I am using CMake to build this. As a minimal example I have the following files: CMakeLists.txt: cmake_minimum_required (VERSION 2.8) project (TEST) add_subdirectory (src) add_subdirectory (test) src/CMakeLists.txt: add_executable (demo main.cpp sqr.cpp) src/sqr.h #ifndef SQR_H #define SQR_H double sqr(double); #endif // SQR_H src/sqr.cpp #include "sqr.h" double sqr(double x) { return x*x; } src/main.cpp - uses sqr, doesn't really matter test/CMakeLists.txt: find_package
  • 如何在Cmake构建系统中使用目录中的所有* .c文件?(How to use all *.c files in a directory with the Cmake build system?)
    问题 我想在目录下找到所有.c文件,并将它们全部添加到SRC文件中,以便在cmake中进行编译。 如何在CMakeList.txt中执行此操作。 对于常规的makefile,我可以创建 SPECIFIED_SRC_FILE = $(foreach d,$(SPECIFIED_SRC_DIRS),$(wildcard $(addprefix $(d)/*,*.c))) 但我无法在CMakeList.txt中找到类似的方法。 回答1 尝试这个: AUX_SOURCE_DIRECTORY 在目录中查找所有源文件。 AUX_SOURCE_DIRECTORY(dir VARIABLE) 收集指定目录中所有源文件的名称,并将列表存储在提供的变量中。 该命令旨在供使用显式模板实例化的项目使用。 模板实例化文件可以存储在“模板”子目录中,并使用此命令自动收集,以避免手动列出所有实例化。 极力使用此命令来避免编写库或可执行目标的源文件列表。 尽管这似乎可行,但是CMake无法生成知道何时添加新源文件的生成系统。 通常,生成的构建系统知道何时需要重新运行CMake,因为修改了CMakeLists.txt文件以添加新的源。 当仅将源代码添加到目录而不修改该文件时,将不得不手动重新运行CMake来生成包含新文件的构建系统。 回答2 好旧的地球如何? FILE(GLOB MyCSources *.c)
  • CPack: Exclude INSTALL commands from subdirectory (googletest directory)
    I'm using CMake for a project and googletest for my test cases. Looking around the internet, it seems to be common practise to just copy the googletest source into a subfolder of your repository and include it with "add_subdirectory(googletest)". I did that. Now I'm using CPack to generate debian packages for my project. Unfortunately, the packages generated by CPack install googletest alongside with my project. This is of course not what I want. Looking in the googletest directory, I found some INSTALL cmake commands there, so it is clear, why it happens. The question is now - how can I avoid