Add option to choose between NSYS and NCU profilers (#28)

* Add option to give nvcc extra arguments

* Add test for nvcc options that changes c++ dialect from c++17 to c++14

* Add make and the english language pack to devcontainer to be able to build the documentation

* Update documentation config to automatically import the current version of the package

* Document new --compiler-args argument

* Improve tests coverage by testing for bad arguments and the error output during a failed compilation

* Add IPython to docs requirements to allow the __version__ import for readthedocs env

* Change devcontainer base image to have the latest CUDA toolkit

* Mock the nsight compute tool with a bash script

* Add test to compile with opencv

* Add new page to documentation that contains a new notebook that explains compiling with external libraries

* Add autodocstring vscode extension to devcontainer

* Add function that modifies the default profiler/compiler arguments to allow reusing them in multiple magic command calls

* Update pylint exceptions

* Update contributing instructions

* Change version from 1.0.3 to 1.1.0 due to adding features in a backward-compatible manner

* Install latest CUDA toolkit on the test runner to pass the OpenCV compilation test

* Install opencv in test runner and update code coverage install

* Add CUDA bin to PATH in test and coverage runners

* Add cuda bin to path variable in .bashrc

* Update way to set environment variable PATH in github action

* Change devcontainer base image back to ubuntu:22.04 to match the environment from the test runner

* Add option to choose between NSYS and NCU profilers

* Add tests for choosing the profiler

* Add isort config to help it find local modules so they are not considered 3rd party libraries

* Replace experimental-string-processing black formatter config with enable-unstable-feature as it was removed in version 24.1.0

* Search for profiling tools executable paths when they are required

* Install dev dependencies in editable mode

* Add documentation for using Nsight Systems instead of the default Nsight Compute profiling tool

* Fix cuda typo

* Mention Nsight Systems in README.md
This commit is contained in:
Cosmin Ștefan Ciocan
2024-03-20 11:42:27 +01:00
committed by GitHub
parent 781ff5b76b
commit 0bddf6a6e6
15 changed files with 293 additions and 89 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
# install developer dependencies # install developer dependencies
pip install .[dev] pip install -e .[dev]
# make sure the developer uses pre-commit hooks # make sure the developer uses pre-commit hooks
pre-commit install pre-commit install
+2 -2
View File
@@ -45,7 +45,7 @@ to own a GPU yourself.
Here are just a few of the things that nvcc4jupyter does well: Here are just a few of the things that nvcc4jupyter does well:
- [Easily run CUDA C++ code](https://nvcc4jupyter.readthedocs.io/en/latest/usage.html#hello-world) - [Easily run CUDA C++ code](https://nvcc4jupyter.readthedocs.io/en/latest/usage.html#hello-world)
- [Profile your code with NVIDIA Nsight Compute](https://nvcc4jupyter.readthedocs.io/en/latest/usage.html#profiling) - [Profile your code with NVIDIA Nsight Compute or Nsight Systems](https://nvcc4jupyter.readthedocs.io/en/latest/usage.html#profiling)
- [Compile your code with external libraries (e.g. OpenCV)](https://nvcc4jupyter.readthedocs.io/en/latest/notebooks.html#compiling-with-external-libraries) - [Compile your code with external libraries (e.g. OpenCV)](https://nvcc4jupyter.readthedocs.io/en/latest/notebooks.html#compiling-with-external-libraries)
- [Share code between different programs in the same notebook / split your code into multiple files for improved readability](https://nvcc4jupyter.readthedocs.io/en/latest/usage.html#groups) - [Share code between different programs in the same notebook / split your code into multiple files for improved readability](https://nvcc4jupyter.readthedocs.io/en/latest/usage.html#groups)
@@ -96,7 +96,7 @@ If not using the devcontainer you need to install the package with the
development dependencies and install the pre-commit hook before commiting any development dependencies and install the pre-commit hook before commiting any
changes: changes:
```bash ```bash
pip install .[dev] pip install -e .[dev]
pre-commit install pre-commit install
``` ```
+25 -5
View File
@@ -21,7 +21,7 @@ Usage
- ``%%cuda``: Compile and run this cell. - ``%%cuda``: Compile and run this cell.
- ``%%cuda -p``: Also runs the Nsight Compute profiler. - ``%%cuda -p``: Also runs the Nsight Compute profiler.
- ``%%cuda -p -a "<SPACE SEPARATED PROFILER ARGS>"``: Also runs the Nsight Compute profiler. - ``%%cuda -p -a "<SPACE SEPARATED PROFILER ARGS>"``: Also runs the Nsight Compute profiler.
- ``%%cude -c "<SPACE SEPARATED COMPILER ARGS"``: Passes additional arguments to "nvcc". - ``%%cuda -c "<SPACE SEPARATED COMPILER ARGS"``: Passes additional arguments to "nvcc".
- ``%%cuda -t``: Outputs the "timeit" built-in magic results. - ``%%cuda -t``: Outputs the "timeit" built-in magic results.
Options Options
@@ -36,15 +36,35 @@ Options
.. _profile: .. _profile:
-p, --profile -p, --profile
Boolean. If set, runs the NVIDIA Nsight Compute profiler whose Boolean. If set, runs the NVIDIA Nsight Compute (or NVIDIA Nsight Systems
output is appended to standard output. if changed via the \-\-profiler option) profiler whose output is appended to
standard output.
.. _profiler:
-l, --profiler
String. Can either be "ncu" (the default) to use NVIDIA Nsight Compute
profiling tool, or "nsys" to use NVIDIA Nsight Systems profiling tool.
.. _profiler_args:
.. _profiler_args: .. _profiler_args:
-a, --profiler-args -a, --profiler-args
String. Optional profiler arguments that can be space separated String. Optional profiler arguments that can be space separated
by wrapping them in double quotes. See all options here: by wrapping them in double quotes. Will be passed to the profiler selected
`Nsight Compute CLI <https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html#command-line-options>`_ by the \-\-profiler option.. See profiler options here:
`Nsight Compute <https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html#command-line-options>`_
or `Nsight Systems <https://docs.nvidia.com/nsight-systems/UserGuide/index.html#command-line-options>`_.
.. _compiler_args:
-c, --compiler-args
String. Optional compiler arguments that can be space separated
by wrapping them in double quotes. They will be passed to "nvcc".
See all options here:
`NVCC Options <https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#nvcc-command-options>`_
.. _compiler_args: .. _compiler_args:
+25 -3
View File
@@ -225,10 +225,11 @@ Profiling
--------- ---------
Another important feature of nvcc4jupyter is its integration with the NVIDIA Another important feature of nvcc4jupyter is its integration with the NVIDIA
Nsight Compute profiler, which you need to make sure is installed and its Nsight Compute / NVIDIA Nsight Systems profilers, which you need to make sure
executable can be found in a directory in your PATH environment variable. are installed and the executables can be found in a directory in your PATH
environment variable.
In order to use it and provide the profiler with custom arguments, simply run: To profile using Nsight Compute with custom arguments:
.. code-block:: c++ .. code-block:: c++
@@ -256,6 +257,27 @@ Running the cell above will compile and execute the vector addition code in the
Compute (SM) Throughput % 1.19 Compute (SM) Throughput % 1.19
----------------------- ------------- ------------ ----------------------- ------------- ------------
To profile using Nsight Systems with custom arguments:
.. code-block:: c++
%cuda_group_run --group "vector_add" --profiler nsys --profile --profiler-args "profile --stats=true"
Running the cell above will compile and execute the vector addition code in the
"vector_add" group and profile it with Nsight Systems. The output will contain
multiple tables, one of which will look similar to this:
.. code-block::
[5/8] Executing 'cuda_api_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ------------- ------------- ----------- ----------- ----------- ----------------------
77.3 200,844,276 1 200,844,276.0 200,844,276.0 200,844,276 200,844,276 0.0 cudaMalloc
22.6 58,594,762 2 29,297,381.0 29,297,381.0 29,153,999 29,440,763 202,772.8 cudaMemcpy
0.1 305,450 1 305,450.0 305,450.0 305,450 305,450 0.0 cudaLaunchKernel
0.0 1,970 1 1,970.0 1,970.0 1,970 1,970 0.0 cuModuleGetLoadingMode
Compiler arguments Compiler arguments
------------------ ------------------
+1 -1
View File
@@ -2,7 +2,7 @@
nvcc4jupyter: CUDA C++ plugin for Jupyter Notebook nvcc4jupyter: CUDA C++ plugin for Jupyter Notebook
""" """
from .parsers import set_defaults # noqa: F401 from .parsers import Profiler, set_defaults # noqa: F401
from .plugin import NVCCPlugin, load_ipython_extension # noqa: F401 from .plugin import NVCCPlugin, load_ipython_extension # noqa: F401
__version__ = "1.1.0" __version__ = "1.1.0"
+36 -6
View File
@@ -3,14 +3,28 @@ Parsers for the CUDA magic commands.
""" """
import argparse import argparse
from typing import Callable, Optional from enum import Enum
from typing import Callable, Optional, Type, TypeVar
class Profiler(Enum):
"""Choice between Nsight Compute and Nsight Systems profilers."""
NCU = "ncu"
NSYS = "nsys"
_default_profiler: Profiler = Profiler.NCU
_default_profiler_args: str = "" _default_profiler_args: str = ""
_default_compiler_args: str = "" _default_compiler_args: str = ""
T = TypeVar("T")
def set_defaults( def set_defaults(
compiler_args: Optional[str] = None, profiler_args: Optional[str] = None profiler: Optional[Profiler] = None,
compiler_args: Optional[str] = None,
profiler_args: Optional[str] = None,
) -> None: ) -> None:
""" """
Set the default values for various arguments of the magic commands. These Set the default values for various arguments of the magic commands. These
@@ -18,17 +32,22 @@ def set_defaults(
to override this behaviour on a cell by cell basis. to override this behaviour on a cell by cell basis.
Args: Args:
profiler: If not None, this value becomes the new default profiler.
Defaults to None.
compiler_args: If not None, this value becomes the new default compiler compiler_args: If not None, this value becomes the new default compiler
config. Defaults to "". config. Defaults to None.
profiler_args: If not None, this value becomes the new default profiler profiler_args: If not None, this value becomes the new default profiler
config. Defaults to "". config. Defaults to None.
""" """
# pylint: disable=global-statement # pylint: disable=global-statement
global _default_profiler
if profiler is not None:
_default_profiler = profiler
global _default_compiler_args global _default_compiler_args
global _default_profiler_args
if compiler_args is not None: if compiler_args is not None:
_default_compiler_args = compiler_args _default_compiler_args = compiler_args
global _default_profiler_args
if profiler_args is not None: if profiler_args is not None:
_default_profiler_args = profiler_args _default_profiler_args = profiler_args
@@ -38,6 +57,11 @@ def str_to_lambda(arg: str) -> Callable[[], str]:
return lambda: arg return lambda: arg
def class_to_lambda(arg: str, cls: Type[T]) -> Callable[[], T]:
"""Convert string value to class and then to lambda"""
return lambda: cls(arg)
def get_parser_cuda() -> argparse.ArgumentParser: def get_parser_cuda() -> argparse.ArgumentParser:
""" """
%%cuda magic command parser. %%cuda magic command parser.
@@ -52,8 +76,14 @@ def get_parser_cuda() -> argparse.ArgumentParser:
parser.add_argument("-t", "--timeit", action="store_true") parser.add_argument("-t", "--timeit", action="store_true")
parser.add_argument("-p", "--profile", action="store_true") parser.add_argument("-p", "--profile", action="store_true")
# --profiler-args and --compiler-args values are lambda functions to allow # the type of the following arguments is a lambda lambda function to allow
# changing the default value at runtime # changing the default value at runtime
parser.add_argument(
"-l",
"--profiler",
type=lambda arg: class_to_lambda(arg, cls=Profiler),
default=lambda: _default_profiler,
)
parser.add_argument( parser.add_argument(
"-a", "-a",
"--profiler-args", "--profiler-args",
+61
View File
@@ -0,0 +1,61 @@
"""
Helper functions relating to file paths.
"""
import os
from glob import glob
from typing import List, Optional
CUDA_SEARCH_PATHS: List[str] = [
"/opt/nvidia/nsight-compute",
"/usr/local/cuda",
"/opt",
"/usr",
]
def is_executable(fpath: str) -> bool:
"""Check if file exists and is executable"""
return os.path.isfile(fpath) and os.access(fpath, os.X_OK)
def which(name: str) -> Optional[str]:
"""Find an executable by name by searching the PATH directories"""
for path_dir in os.environ.get("PATH", "").split(os.pathsep):
exec_path = os.path.join(path_dir, name)
if is_executable(exec_path):
return exec_path
return None
def find_executable(
name: str, search_paths: Optional[List[str]] = None
) -> Optional[str]:
"""
Find an executable, either by searching in the directories of the PATH
environment variable or, if that did not work, by searching recursively
in directories a list given as parameter.
Args:
name: The name of the executable to be found.
search_paths: If None, only executables that are available from PATH
will be found. Otherwise, will recursively search these
directories. Defaults to None.
Returns:
The path to the executable if it is found, and None otherwise.
"""
if search_paths is None:
search_paths = []
which_path = which(name)
if which_path is not None:
return which_path
for search_path in search_paths:
search_path = os.path.abspath(search_path)
search_path = os.path.join(search_path, f"**/{name}")
for exec_path in glob(search_path, recursive=True):
return exec_path
return None
+56 -10
View File
@@ -9,13 +9,20 @@ import shutil
import subprocess import subprocess
import tempfile import tempfile
import uuid import uuid
from typing import List, Optional from typing import Dict, List, Optional
# pylint: disable=import-error # pylint: disable=import-error
from IPython.core.interactiveshell import InteractiveShell from IPython.core.interactiveshell import InteractiveShell
from IPython.core.magic import Magics, cell_magic, line_magic, magics_class from IPython.core.magic import Magics, cell_magic, line_magic, magics_class
from . import parsers from .parsers import (
Profiler,
get_parser_cuda,
get_parser_cuda_group_delete,
get_parser_cuda_group_run,
get_parser_cuda_group_save,
)
from .path_utils import CUDA_SEARCH_PATHS, find_executable
DEFAULT_EXEC_FNAME = "cuda_exec.out" DEFAULT_EXEC_FNAME = "cuda_exec.out"
SHARED_GROUP_NAME = "shared" SHARED_GROUP_NAME = "shared"
@@ -37,14 +44,19 @@ class NVCCPlugin(Magics):
super().__init__(shell) super().__init__(shell)
self.shell: InteractiveShell # type hint not provided by parent class self.shell: InteractiveShell # type hint not provided by parent class
self.parser_cuda = parsers.get_parser_cuda() self.parser_cuda = get_parser_cuda()
self.parser_cuda_group_save = parsers.get_parser_cuda_group_save() self.parser_cuda_group_save = get_parser_cuda_group_save()
self.parser_cuda_group_delete = parsers.get_parser_cuda_group_delete() self.parser_cuda_group_delete = get_parser_cuda_group_delete()
self.parser_cuda_group_run = parsers.get_parser_cuda_group_run() self.parser_cuda_group_run = get_parser_cuda_group_run()
self.workdir = tempfile.mkdtemp() self.workdir = tempfile.mkdtemp()
print(f'Source files will be saved in "{self.workdir}".') print(f'Source files will be saved in "{self.workdir}".')
self.profiler_paths: Dict[Profiler, Optional[str]] = {
Profiler.NCU: None,
Profiler.NSYS: None,
}
def _save_source( def _save_source(
self, source_name: str, source_code: str, group_name: str self, source_name: str, source_code: str, group_name: str
) -> None: ) -> None:
@@ -135,11 +147,42 @@ class NVCCPlugin(Magics):
return executable_fpath return executable_fpath
def _run( def _get_profiler_path(self, profiler: Profiler) -> str:
"""
Get the path of the executable of a given profiling tool. Searches
the directories of the PATH environment variable and some extra
directories where CUDA is usually installed.
Args:
profiler: The profiler whose executable should be found.
Raises:
RuntimeError: If the profiler executable could not be found.
Returns:
The file path of the executable.
"""
profiler_path = self.profiler_paths[profiler]
if profiler_path is not None:
return profiler_path
profiler_path = find_executable(profiler.value, CUDA_SEARCH_PATHS)
if profiler_path is None:
raise RuntimeError(
f'Could not find the "{profiler.value}" profiling tool.'
" Consider searching for where it is installed and adding its"
" directory to the PATH environment variable."
)
self.profiler_paths[profiler] = profiler_path
return profiler_path
def _run( # pylint: disable=too-many-arguments
self, self,
exec_fpath: str, exec_fpath: str,
timeit: bool = False, timeit: bool = False,
profile: bool = False, profile: bool = False,
profiler: Profiler = Profiler.NCU,
profiler_args: str = "", profiler_args: str = "",
) -> str: ) -> str:
""" """
@@ -150,8 +193,9 @@ class NVCCPlugin(Magics):
timeit: If True, returns the result of the "timeit" magic instead timeit: If True, returns the result of the "timeit" magic instead
of the standard output of the CUDA process. Defaults to False. of the standard output of the CUDA process. Defaults to False.
profile: If True, the executable is profiled with NVIDIA Nsight profile: If True, the executable is profiled with NVIDIA Nsight
Compute profiling tool and its output is added to stdout. Compute or NVIDIA Nsight Systems and the profiling output is
Defaults to False. added to stdout. Defaults to False.
profiler: The profiling tool to use.
profiler_args: The profiler arguments used to customize the profiler_args: The profiler arguments used to customize the
information gathered by it and its overall behaviour. Defaults information gathered by it and its overall behaviour. Defaults
to an empty string. to an empty string.
@@ -173,7 +217,8 @@ class NVCCPlugin(Magics):
else: else:
run_args = [] run_args = []
if profile: if profile:
run_args.extend(["ncu"] + profiler_args.split()) profiler_path = self._get_profiler_path(profiler)
run_args.extend([profiler_path] + profiler_args.split())
run_args.append(exec_fpath) run_args.append(exec_fpath)
output = subprocess.check_output( output = subprocess.check_output(
run_args, stderr=subprocess.STDOUT run_args, stderr=subprocess.STDOUT
@@ -194,6 +239,7 @@ class NVCCPlugin(Magics):
exec_fpath=exec_fpath, exec_fpath=exec_fpath,
timeit=args.timeit, timeit=args.timeit,
profile=args.profile, profile=args.profile,
profiler=args.profiler(),
profiler_args=args.profiler_args(), profiler_args=args.profiler_args(),
) )
except subprocess.CalledProcessError as e: except subprocess.CalledProcessError as e:
+3 -1
View File
@@ -69,6 +69,7 @@ exclude_lines = [
[tool.isort] [tool.isort]
profile = "black" profile = "black"
src_paths = ["nvcc4jupyter"] # tells isort where to find local modules to not consider them 3rd party libraries
[tool.bandit] [tool.bandit]
exclude_dirs = ["build","dist","tests","scripts"] exclude_dirs = ["build","dist","tests","scripts"]
@@ -82,7 +83,8 @@ skips = ["B101", "B311", "B404", "B603"]
[tool.black] [tool.black]
line-length = 79 line-length = 79
fast = true fast = true
experimental-string-processing = true preview = true
enable-unstable-feature = ["string_processing"]
[tool.coverage.run] [tool.coverage.run]
branch = true branch = true
+13
View File
@@ -1,9 +1,11 @@
import argparse
import glob import glob
import os import os
import pytest import pytest
from IPython.core.interactiveshell import InteractiveShell from IPython.core.interactiveshell import InteractiveShell
from nvcc4jupyter.parsers import Profiler
from nvcc4jupyter.plugin import NVCCPlugin from nvcc4jupyter.plugin import NVCCPlugin
@@ -70,3 +72,14 @@ def multiple_source_fpaths(fixtures_path: str):
pattern_h = os.path.join(fixtures_path, "multiple_files", "*.h") pattern_h = os.path.join(fixtures_path, "multiple_files", "*.h")
pattern_cu = os.path.join(fixtures_path, "multiple_files", "*.cu") pattern_cu = os.path.join(fixtures_path, "multiple_files", "*.cu")
return list(glob.glob(pattern_h)) + list(glob.glob(pattern_cu)) return list(glob.glob(pattern_h)) + list(glob.glob(pattern_cu))
@pytest.fixture(scope="session")
def default_args():
return argparse.Namespace(
timeit=False,
profile=True,
profiler=lambda: Profiler.NCU,
profiler_args=lambda: "",
compiler_args=lambda: "",
)
+2 -2
View File
@@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
echo "[NCU]"
# this is a mock of nsight compute cli tool that just executes the program # this is a mock of nsight compute cli tool that just executes the program
# given as the last argument # given as the last argument
"${@: -1}" "${@: -1}"
echo "==WARNING== No kernels were profiled"
Vendored Executable
+7
View File
@@ -0,0 +1,7 @@
#!/bin/bash
echo "[NSYS]"
# this is a mock of nsight systems cli tool that just executes the program
# given as the last argument
"${@: -1}"
+3
View File
@@ -0,0 +1,3 @@
#!/bin/bash
echo "This is just used to test the path_utils.find_executable function"
+16
View File
@@ -0,0 +1,16 @@
import os
from nvcc4jupyter.path_utils import find_executable
def test_which():
assert find_executable("ls") == "/usr/bin/ls"
def test_find_executable(fixtures_path: str):
exec_path = find_executable("searchforme", [fixtures_path])
assert exec_path is not None
exec_dir, exec_fname = os.path.split(exec_path)
assert exec_fname == "searchforme"
assert os.path.basename(exec_dir) == "scripts"
+42 -58
View File
@@ -1,29 +1,26 @@
import argparse
import math import math
import os import os
import re import re
import shutil import shutil
import subprocess import subprocess
from argparse import ArgumentParser, Namespace
from copy import deepcopy
from typing import List from typing import List
import pytest import pytest
from nvcc4jupyter.parsers import get_parser_cuda, set_defaults from nvcc4jupyter.parsers import Profiler, get_parser_cuda, set_defaults
from nvcc4jupyter.plugin import NVCCPlugin from nvcc4jupyter.plugin import NVCCPlugin
def check_profiler_output(output: str): def check_profiler_output(output: str, profiler: str = "[NCU]"):
# the profiler output will be a line of "Hello World!" along with some # the output from the profiler will first be a line containing only
# warning lines which start with "==WARNING==" # "[NCU]" or "[NSYS]" depending on what profiler was used and another
# line containing the string "Hello World!"
lines = output.strip().split("\n") lines = output.strip().split("\n")
warn_count = 0 assert len(lines) >= 2
for line in lines: assert lines[0] == profiler
if not line.startswith("==WARNING=="): assert lines[1] == "Hello World!"
assert line == "Hello World!"
else:
warn_count += 1
assert warn_count >= 1
assert warn_count == len(lines) - 1
def copy_source_to_group( def copy_source_to_group(
@@ -46,7 +43,7 @@ def before_all(scripts_path: str):
@pytest.fixture(autouse=True, scope="function") @pytest.fixture(autouse=True, scope="function")
def before_each(plugin: NVCCPlugin): def before_each(plugin: NVCCPlugin):
# BEFORE TESTS # BEFORE TESTS
set_defaults(compiler_args="", profiler_args="") set_defaults(profiler=Profiler.NCU, compiler_args="", profiler_args="")
shutil.rmtree(plugin.workdir, ignore_errors=True) shutil.rmtree(plugin.workdir, ignore_errors=True)
yield yield
# AFTER TESTS # AFTER TESTS
@@ -101,6 +98,7 @@ def test_compile(
def test_compile_args( def test_compile_args(
plugin: NVCCPlugin, plugin: NVCCPlugin,
compiler_cpp_17_fpath: str, compiler_cpp_17_fpath: str,
default_args: Namespace,
): ):
gname = "test_compile_args" gname = "test_compile_args"
copy_source_to_group(compiler_cpp_17_fpath, gname, plugin.workdir) copy_source_to_group(compiler_cpp_17_fpath, gname, plugin.workdir)
@@ -112,21 +110,16 @@ def test_compile_args(
with pytest.raises(subprocess.CalledProcessError): with pytest.raises(subprocess.CalledProcessError):
exec_fpath = plugin._compile(gname, compiler_args="--std c++14") exec_fpath = plugin._compile(gname, compiler_args="--std c++14")
output = plugin._compile_and_run( args = deepcopy(default_args)
group_name=gname, args.compiler_args = lambda: "--std c++14"
args=argparse.Namespace( output = plugin._compile_and_run(group_name=gname, args=args)
timeit=False,
profile=True,
profiler_args=lambda: "",
compiler_args=lambda: "--std c++14",
),
)
assert "errors detected in the compilation of" in output assert "errors detected in the compilation of" in output
def test_compile_opencv( def test_compile_opencv(
plugin: NVCCPlugin, plugin: NVCCPlugin,
compiler_opencv_fpath: str, compiler_opencv_fpath: str,
default_args: Namespace,
): ):
gname = "test_compile_opencv" gname = "test_compile_opencv"
copy_source_to_group(compiler_opencv_fpath, gname, plugin.workdir) copy_source_to_group(compiler_opencv_fpath, gname, plugin.workdir)
@@ -134,23 +127,14 @@ def test_compile_opencv(
# check that "pkg-config" exists # check that "pkg-config" exists
assert subprocess.check_call(["which", "pkg-config"]) == 0 assert subprocess.check_call(["which", "pkg-config"]) == 0
pkg_config_args = ["pkg-config", "--cflags", "--libs", "opencv4"]
opencv_compile_options = ( opencv_compile_options = (
subprocess.check_output( subprocess.check_output(args=pkg_config_args).decode().strip()
args=["pkg-config", "--cflags", "--libs", "opencv4"]
)
.decode()
.strip()
) )
output = plugin._compile_and_run( args = deepcopy(default_args)
group_name=gname, args.compiler_args = lambda: opencv_compile_options
args=argparse.Namespace( output = plugin._compile_and_run(group_name=gname, args=args)
timeit=False,
profile=True,
profiler_args=lambda: "",
compiler_args=lambda: opencv_compile_options,
),
)
assert "General configuration for OpenCV" in output assert "General configuration for OpenCV" in output
@@ -199,7 +183,9 @@ def test_run_profile(plugin: NVCCPlugin, sample_cuda_fpath: str):
def test_compile_and_run_multiple_files( def test_compile_and_run_multiple_files(
plugin: NVCCPlugin, multiple_source_fpaths: List[str] plugin: NVCCPlugin,
multiple_source_fpaths: List[str],
default_args: Namespace,
): ):
""" """
Compiles and executes 3 cuda source files from Compiles and executes 3 cuda source files from
@@ -208,20 +194,14 @@ def test_compile_and_run_multiple_files(
gname = "test_compile_and_run_multiple_files" gname = "test_compile_and_run_multiple_files"
for fpath in multiple_source_fpaths: for fpath in multiple_source_fpaths:
copy_source_to_group(fpath, gname, plugin.workdir) copy_source_to_group(fpath, gname, plugin.workdir)
output = plugin._compile_and_run( output = plugin._compile_and_run(group_name=gname, args=default_args)
group_name=gname,
args=argparse.Namespace(
timeit=False,
profile=True,
profiler_args=lambda: "",
compiler_args=lambda: "",
),
)
check_profiler_output(output) check_profiler_output(output)
def test_compile_and_run_multiple_files_shared( def test_compile_and_run_multiple_files_shared(
plugin: NVCCPlugin, multiple_source_fpaths: List[str] plugin: NVCCPlugin,
multiple_source_fpaths: List[str],
default_args: Namespace,
): ):
""" """
Compiles and executes 3 cuda source files from Compiles and executes 3 cuda source files from
@@ -236,20 +216,12 @@ def test_compile_and_run_multiple_files_shared(
copy_source_to_group(fpath, gname, plugin.workdir) copy_source_to_group(fpath, gname, plugin.workdir)
else: else:
copy_source_to_group(fpath, "shared", plugin.workdir) copy_source_to_group(fpath, "shared", plugin.workdir)
output = plugin._compile_and_run( output = plugin._compile_and_run(group_name=gname, args=default_args)
group_name=gname,
args=argparse.Namespace(
timeit=False,
profile=True,
profiler_args=lambda: "",
compiler_args=lambda: "",
),
)
check_profiler_output(output) check_profiler_output(output)
def test_read_args(plugin: NVCCPlugin): def test_read_args(plugin: NVCCPlugin):
parser = argparse.ArgumentParser() parser = ArgumentParser()
parser.add_argument("-a", type=str, required=True) parser.add_argument("-a", type=str, required=True)
parser.add_argument("-b", type=float, required=True) parser.add_argument("-b", type=float, required=True)
args = plugin._read_args( args = plugin._read_args(
@@ -292,6 +264,18 @@ def test_magic_cuda(
check_profiler_output(capsys.readouterr().out) check_profiler_output(capsys.readouterr().out)
def test_magic_cuda_set_default_profiler(
capsys,
plugin: NVCCPlugin,
sample_cuda_code: str,
sample_magic_cu_line: str,
):
# set the default profiler to Nsight Systems
set_defaults(profiler=Profiler.NSYS)
plugin.cuda(sample_magic_cu_line, sample_cuda_code)
check_profiler_output(capsys.readouterr().out, profiler="[NSYS]")
def test_magic_cuda_bad_args( def test_magic_cuda_bad_args(
capsys, capsys,
plugin: NVCCPlugin, plugin: NVCCPlugin,