Adding an utility command line to streamline profiling

Author: nyamsprod

.php-cs-fixer.php

@@ -3,6 +3,7 @@
 $finder = PhpCsFixer\Finder::create()
     ->in(__DIR__.'/src')
     ->in(__DIR__.'/benchmark')
+    ->in(__DIR__.'/bin')
 ;
 
 $config = new PhpCsFixer\Config();

CHANGELOG.md

@@ -8,6 +8,11 @@ All Notable changes to `bakame/aide-profiler` will be documented in this file.
 
 - `Statistics` class
 - `Report` class and the `Profiler::report` method.
+- `ConsoleTableExporter::exportReport`
+- `ConsoleTableExporter::exportStatistics`
+- `ConsoleTableExporter::exportMetrics`
+- `JsonExporter`
+- `Profile` attribute, the `PathProfiler` class and the `phpProfiler` command to ease profiling
 
 ### Fixed
 

README.md

@@ -10,7 +10,8 @@
 [![Sponsor development of this project](https://img.shields.io/badge/sponsor%20this%20package-%E2%9D%A4-ff69b4.svg?style=flat-square)](https://github.com/sponsors/nyamsprod)
 
 A minimalist profiler for PHP. The profiler is embeddable, multi-metric, and framework-agnostic 
-It fills the gap between a basic timer and full-blown profilers like [Xdebug](https://xdebug.org/) or [Blackfire](https://www.blackfire.io/).
+It fills the gap between a basic timer and full-blown profilers like: [PHPBench](https://phpbench.readthedocs.io/en/latest/), 
+[Xdebug](https://xdebug.org/), [Blackfire](https://www.blackfire.io/).
 
 ## Installation
 
@@ -121,7 +122,7 @@ and formatted.
 use Bakame\Aide\Profiler\Profiler;
 
 // you create a new Profiler by passing the callback you want to profile
-$report = Profiler::report($service->calculateHeavyStuff(...));
+$report = Profiler::report($service->calculateHeavyStuff(...), 500);
 
 // Access the raw statistical metrics
 $report->executionTime->minimum; // Minimum execution time (as float|int, in nanoseconds)
@@ -532,6 +533,25 @@ echo json_encode($marker), PHP_EOL;
 ```
 See a [sample marker JSON output](./examples/marker-sample.json) for a complete structure.
 
+In order to facilitate JSON export, the package has a dedicated `JsonExporter` class
+which will be able to store the generated json in the specified location. It supports
+streams, string path and `SplFileInfo` objects.
+
+```php
+use Bakame\Aide\Profiler\JsonExporter;
+use Bakame\Aide\Profiler\Profiler;
+
+$report = Profiler::report($service->calculateHeavyStuff(...), 500);
+$exporter = new JsonExporter('path/to/store/the/profile.json', JSON_PRETTY_PRINT|JSON_BIGINT_AS_STRING);
+$exporter->exportReport($report);
+```
+The report will be stored in the designated location.
+
+> [!IMPORTANT]  
+> If you try to store multiple export in the same file (specified by a string)
+> They will get overwritten and only the last export will be stored.
+> To get the data appended provide an already open `resource` or `SplFileObject`.
+
 #### CLI
 
 If you have the `symfony\console` package installed in your application, you can export
@@ -610,6 +630,138 @@ $exporter->exportProfilter($profiler);
 
 Remember to change the `$tracerProvider` to connect to your own environment and server.
 
+### CLI command
+
+A CLI Command is available to allow you to benchmark PHP **functions and methods** located in a specific file or directory using the custom `#[Profile]` attribute.
+
+This is especially useful for:
+
+- Automating performance regressions in CI pipelines
+- Profiling code outside the context of an application
+
+#### Usage
+
+```bash
+php bin/phpProfiler --path=your/script.php [--output=cli|json] [--info] [--help]
+```
+
+| Option              | Description                                                                   |
+|---------------------|-------------------------------------------------------------------------------|
+| `--path[=PATH]`     | **(Required)** Path to the file to scan for profiled functions and methods.   |
+| `--output[=OUTPUT]` | Output format: either `json` or `cli` (default) table.                        |
+| `-i`, `--info`      | Additionally display system-level profiling metadata (PHP version, CPU, etc). |
+| `-h`, `--help`      | Show help for the command.                                                    |
+
+#### Example
+
+let's assume you have the following file located in `/path/profiler/test.php`.
+
+```php
+<?php
+
+declare(strict_types=1);
+
+namespace Foobar\Baz;
+
+use Bakame\Aide\Profiler\Profile;
+use function random_int;
+use function usleep;
+
+require 'vendor/autoload.php';
+
+trait TimerTrait {
+    #[Profile(type: Profile::METRICS, iterations: 10)]
+    private function test() : int {
+        usleep(100);
+
+        return random_int(1, 100);
+    }
+}
+
+enum Foobar
+{
+    use TimerTrait;
+
+    case Foobar;
+}
+
+#[Profile(type: Profile::REPORT, iterations: 20, warmup: 2)]
+function test() : int {
+    usleep(100);
+
+    return random_int(1, 100);
+}
+```
+If you run the following command:
+
+```bash
+php bin/phpProfiler --path=/path/profiler/test.php
+```
+It will output 2 console tables:
+
+```bash
+PHPProfiler 0.11.0 by Ignace Nyamagana Butera and contributors.
+
+Runtime: PHP 8.3.23
+Platform:    Linux
+
+Report for the function Foobar\Baz\test located in /path/profiler/test.php called 20 times
++------------------------+---------------+------------+------------+--------------+------------+-----------+------------+------------+----------+-----------+
+| Metric                 | Nb Iterations | Min Value  | Max Value  | Median Value | Sum        | Range     | Average    | Variance   | Std Dev  | Coef Var  |
++------------------------+---------------+------------+------------+--------------+------------+-----------+------------+------------+----------+-----------+
+| CPU Time               | 20            | 7.000 µs   | 32.000 µs  | 8.000 µs     | 183.000 µs | 25.000 µs | 9.150 µs   | 28.128 μs² | 5.304 µs | 57.9621 % |
+| Execution Time         | 20            | 132.125 µs | 158.208 µs | 133.292 µs   | 2.701 ms   | 26.083 µs | 135.029 µs | 32.436 μs² | 5.695 µs | 4.2178 %  |
+| Memory Usage           | 20            | 1.031 KB   | 1.031 KB   | 1.031 KB     | 20.625 KB  | 0.000 B   | 1.031 KB   | 0.000 B²   | 0.000 B  | 0.0000 %  |
+| Peak Memory Usage      | 20            | 0.000 B    | 0.000 B    | 0.000 B      | 0.000 B    | 0.000 B   | 0.000 B    | 0.000 B²   | 0.000 B  | 0.0000 %  |
+| Real Memory Usage      | 20            | 0.000 B    | 0.000 B    | 0.000 B      | 0.000 B    | 0.000 B   | 0.000 B    | 0.000 B²   | 0.000 B  | 0.0000 %  |
+| Real Peak Memory Usage | 20            | 0.000 B    | 0.000 B    | 0.000 B      | 0.000 B    | 0.000 B   | 0.000 B    | 0.000 B²   | 0.000 B  | 0.0000 %  |
++------------------------+---------------+------------+------------+--------------+------------+-----------+------------+------------+----------+-----------+
+Average metrics for the method Foobar\Baz\Foobar::test located in /path/profiler/test.php called 10 times
++------------------------------------+
+|         Execution Time: 140.213 µs |
+|               CPU Time: 11.700 µs  |
+|           Memory Usage: 1.0 KB     |
+|      Real Memory Usage: 0.0 B      |
+|      Peak Memory Usage: 0.0 B      |
+| Real Peak Memory Usage: 0.0 B      |
++------------------------------------+
+```
+
+- one about the full report on the function `test` (this is equivalent as using `Profiler::report`)
+- the other about the average metrics for the `Foobar::test` method.  (this is equivalent as using `Profiler::metrics`)
+
+The `#[Profile]` attribute exposes the same arguments as the `Profiler` methods:
+
+- `iterations`: Number of times to execute the function for statistical significance.
+- `warmup`: (Optional) Number of warmup iterations before measuring.
+- `type`: Either `Profile::METRICS` or `Profile::REPORT`; To determine if you want the `Profiler::report` or the `Profiler::metrics` output.
+
+#### Notes
+
+The command line supports **function-level** and **method-level** profiling, including methods defined
+via traits, even inside Enums.
+
+- Functions or methods without a `#[Profile]` attribute will be ignored.
+- Functions or methods with arguments will also be ignored.
+
+All required dependencies should be loaded in the target file (use `require`, `include` or Composer autoload).
+
+#### Integration into CI
+
+You can run the profiler command in your CI pipelines to detect regressions or performance anomalies.
+
+```yaml
+- name: Run Profiler
+  run: php bin/phpProfiler --path=/path/profiler/test.php --output=json
+```
+
+> [!IMPORTANT]  
+> The command line requires `symfony\console` and the `psr\log` interfaces to work.
+
+> [!CAUTION]  
+> The command line can scan your full codebase if you specify a directory instead of a path. But
+> favor the cli output as the json output will not return a valid json file.
+
 ### Helpers
 
 #### Environment

bin/phpProfiler

@@ -0,0 +1,19 @@
+#!/usr/bin/env php
+<?php
+
+declare(strict_types=1);
+
+use Bakame\Aide\Profiler\Console\Command;
+
+if (is_file(__DIR__ . '/../vendor/autoload.php')) {
+    require __DIR__ . '/../vendor/autoload.php';
+} elseif (is_file(__DIR__ . '/../../../autoload.php')) {
+    require __DIR__ . '/../../../autoload.php';
+} else {
+    fwrite(STDERR, 'Cannot find the vendor directory, have you executed composer install?' . PHP_EOL);
+    fwrite(STDERR,'See https://getcomposer.org to get Composer.' . PHP_EOL);
+
+    exit(1);
+}
+
+Command::run();

composer.json

@@ -50,6 +50,9 @@
             "benchmark/Benchmark.php"
         ]
     },
+    "bin": [
+        "bin/phpProfiler"
+    ],
     "scripts": {
         "phpcs": "php-cs-fixer fix -vvv --diff --dry-run --allow-risky=yes --ansi",
         "phpcs:fix": "php-cs-fixer fix -vvv --allow-risky=yes --ansi",
@@ -71,7 +74,7 @@
     },
     "suggest": {
         "psr/log": "to log the profiling process",
-        "symfony/console": "to render the profiler in your CLI command",
+        "symfony/console": "to use the CLI command",
         "open-telemetry/exporter-otlp": "to export the profiler results to an opentelemetry compatible server"
     },
     "extra": {

phpunit.xml

@@ -1,5 +1,14 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.3/phpunit.xsd" bootstrap="vendor/autoload.php" backupGlobals="false" colors="true" processIsolation="false" stopOnFailure="false" cacheDirectory=".phpunit.cache" backupStaticProperties="false">
+<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.3/phpunit.xsd"
+         bootstrap="vendor/autoload.php"
+         backupGlobals="false"
+         colors="true"
+         processIsolation="false"
+         stopOnFailure="false"
+         cacheDirectory=".phpunit.cache"
+         backupStaticProperties="false"
+>
   <coverage>
     <report>
       <clover outputFile="build/clover.xml"/>

src/Console/Command.php

@@ -0,0 +1,136 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Bakame\Aide\Profiler\Console;
+
+use Bakame\Aide\Profiler\ConsoleTableExporter;
+use Bakame\Aide\Profiler\Environment;
+use Bakame\Aide\Profiler\PathProfiler;
+use Bakame\Aide\Profiler\Version;
+use RuntimeException;
+use Symfony\Component\Console\Output\OutputInterface;
+use Symfony\Component\Console\Output\StreamOutput;
+use Throwable;
+
+use function class_exists;
+use function fwrite;
+
+use const JSON_PRETTY_PRINT;
+use const PHP_EOL;
+use const PHP_SAPI;
+use const STDERR;
+use const STDOUT;
+
+final class Command
+{
+    public const SUCCESS = 0;
+    public const ERROR = 1;
+
+    public static function run(): never
+    {
+        if (!class_exists(StreamOutput::class)) {
+            fwrite(STDERR, 'The symfony/console package is required to use the command line.'.PHP_EOL);
+
+            exit(self::ERROR);
+        }
+
+        if ('cli' !== PHP_SAPI) {
+            fwrite(STDERR, 'This script must be run from the command line.'.PHP_EOL);
+
+            exit(self::ERROR);
+        }
+
+        (new self(new StreamOutput(STDOUT), new StreamOutput(STDERR)))->handle();
+    }
+
+    public function __construct(private readonly OutputInterface $stdout, private readonly OutputInterface $stderr)
+    {
+    }
+
+    public function handle(): never
+    {
+        exit($this->execute(Input::fromCli()));
+    }
+
+    public function execute(Input $options): int
+    {
+        if ($options->showVersion) {
+            $this->stdout->writeln('<info>'.Version::full().'</info>');
+
+            return self::SUCCESS;
+        }
+
+        if (Input::CLI_FORMAT === $options->outputFormat) {
+            $this->stdout->writeln($this->header());
+        }
+
+        if ($options->showHelp) {
+            $this->stdout->writeln($this->helpText());
+
+            return self::SUCCESS;
+        }
+
+        $environment = Environment::current();
+        if ($options->showInfo) {
+            (new ConsoleTableExporter($this->stdout))->exportEnvironment($environment);
+            $this->stdout->writeln('');
+        }
+
+        if (null === $options->path) {
+            if ($options->showInfo) {
+                return self::SUCCESS;
+            }
+            $this->stderr->writeln('<error> Please specify a valid path. </error>');
+            $this->stdout->writeln($this->helpText());
+
+            return self::ERROR;
+        }
+
+        if (Input::CLI_FORMAT === $options->outputFormat && !$options->showInfo) {
+            $this->stdout->writeln('<fg=green>Runtime:</> PHP '.$environment->phpVersion.' <fg=green>OS:</> '.$environment->os.' <fg=green>Memory Limit:</> '.$environment->rawMemoryLimit);
+            $this->stdout->writeln('');
+        }
+
+        try {
+            (match ($options->outputFormat) {
+                Input::JSON_FORMAT => PathProfiler::forJson(STDOUT, JSON_PRETTY_PRINT, new Logger($this->stderr)),
+                Input::CLI_FORMAT => PathProfiler::forConsole($this->stdout, new Logger($this->stderr)),
+                default => throw new RuntimeException('Unknown output format: '.$options->outputFormat),
+            })->handle($options->path);
+
+            return self::SUCCESS;
+        } catch (Throwable $e) {
+            $this->stderr->writeln('<error> Execution Error: '.$e->getMessage().'</error>');
+
+            return self::ERROR;
+        }
+    }
+
+    private function header(): string
+    {
+        $version = Version::full();
+
+        return <<<HELP
+<fg=green>PHPProfiler $version</><fg=yellow> by Ignace Nyamagana Butera and contributors.</>
+
+HELP;
+    }
+
+    private function helpText(): string
+    {
+        return <<<'HELP'
+<fg=yellow>Description:</>
+  Simple command to profile functions and methods located in a specific file.
+
+<fg=yellow>Usage:</>
+  phpProfiler --path[=PATH] --output[=OUTPUT] [--info] [--help]
+
+<fg=yellow>Options:</>
+<fg=green>  --path[=PATH]</>       The path to the file to parse containing class and methods to profile
+<fg=green>  --output[=OUTPUT]</>   Output the data as Json or CLI table
+<fg=green>  -h, --help</>          Display help for the given command
+<fg=green>  -i, --info</>          Additionally display the system general information
+HELP;
+    }
+}