1PYINSTRUMENT(1) pyinstrument PYINSTRUMENT(1)
2
3
4
6 pyinstrument - pyinstrument 4.4.0
7
9 Screenshot
10
11 Pyinstrument is a Python profiler. A profiler is a tool to help you op‐
12 timize your code - make it faster. To get the biggest speed increase
13 you should focus on the slowest part of your program. Pyinstrument
14 helps you find it!
15 ☕️ Not sure where to start? Check out this video tutorial from
16 calmcode.io!
17
19 Installation
20 pip install pyinstrument
21
22
23 Pyinstrument supports Python 3.7+.
24
25 Profile a Python script
26 Call Pyinstrument directly from the command line. Instead of writing
27 python script.py, type pyinstrument script.py. Your script will run as
28 normal, and at the end (or when you press ^C), Pyinstrument will output
29 a colored summary showing where most of the time was spent.
30
31 Here are the options you can use:
32
33 Usage: pyinstrument [options] scriptfile [arg] ...
34
35 Options:
36 --version show program's version number and exit
37 -h, --help show this help message and exit
38 --load-prev=ID instead of running a script, load a previous report
39 -m MODULE_NAME run library module as a script, like 'python -m
40 module'
41 --from-path (POSIX only) instead of the working directory, look
42 for scriptfile in the PATH environment variable
43 -o OUTFILE, --outfile=OUTFILE
44 save to <outfile>
45 -r RENDERER, --renderer=RENDERER
46 how the report should be rendered. One of: 'text',
47 'html', 'json', 'speedscope', or python import path
48 to a renderer class
49 -t, --timeline render as a timeline - preserve ordering and don't
50 condense repeated calls
51 --hide=EXPR glob-style pattern matching the file paths whose
52 frames to hide. Defaults to '*/lib/*'.
53 --hide-regex=REGEX regex matching the file paths whose frames to hide.
54 Useful if --hide doesn't give enough control.
55 --show=EXPR glob-style pattern matching the file paths whose
56 frames to show, regardless of --hide or --hide-regex.
57 For example, use --show '*/<library>/*' to show frames
58 within a library that would otherwise be hidden.
59 --show-regex=REGEX regex matching the file paths whose frames to always
60 show. Useful if --show doesn't give enough control.
61 --show-all show everything
62 --unicode (text renderer only) force unicode text output
63 --no-unicode (text renderer only) force ascii text output
64 --color (text renderer only) force ansi color text output
65 --no-color (text renderer only) force no color text output
66
67
68 Protip: -r html will give you a interactive profile report as HTML -
69 you can really explore this way!
70
71 Profile a specific chunk of code
72 Pyinstrument also has a Python API. Just surround your code with Pyin‐
73 strument, like this:
74
75 from pyinstrument import Profiler
76
77 profiler = Profiler()
78 profiler.start()
79
80 # code you want to profile
81
82 profiler.stop()
83
84 profiler.print()
85
86
87 If you get "No samples were recorded." because your code executed in
88 under 1ms, hooray! If you still want to instrument the code, set an in‐
89 terval value smaller than the default 0.001 (1 millisecond) like this:
90
91 profiler = Profiler(interval=0.0001)
92 ...
93
94
95 Experiment with the interval value to see different depths, but keep in
96 mind that smaller intervals could affect the performance overhead of
97 profiling.
98
99 Protip: To explore the profile in a web browser, use
100 profiler.open_in_browser(). To save this HTML for later, use
101 profiler.output_html().
102
103 Profile code in Jupyter/IPython
104 Via IPython magics, you can profile a line or a cell in IPython or
105 Jupyter.
106
107 Example:
108
109 %load_ext pyinstrument
110
111
112 %%pyinstrument
113 import time
114
115 def a():
116 b()
117 c()
118 def b():
119 d()
120 def c():
121 d()
122 def d():
123 e()
124 def e():
125 time.sleep(1)
126 a()
127
128
129 To customize options, see %%pyinstrument??.
130
131 Profile a web request in Django
132 To profile Django web requests, add pyinstrument.middleware.Profiler‐
133 Middleware to MIDDLEWARE in your settings.py.
134
135 Once installed, add ?profile to the end of a request URL to activate
136 the profiler. Your request will run as normal, but instead of getting
137 the response, you'll get pyinstrument's analysis of the request in a
138 web page.
139
140 If you're writing an API, it's not easy to change the URL when you want
141 to profile something. In this case, add PYINSTRUMENT_PROFILE_DIR =
142 'profiles' to your settings.py. Pyinstrument will profile every request
143 and save the HTML output to the folder profiles in your working direc‐
144 tory.
145
146 If you want to show the profiling page depending on the request you can
147 define PYINSTRUMENT_SHOW_CALLBACK as dotted path to a function used for
148 determining whether the page should show or not. You can provide your
149 own function callback(request) which returns True or False in your set‐
150 tings.py.
151
152 def custom_show_pyinstrument(request):
153 return request.user.is_superuser
154
155
156 PYINSTRUMENT_SHOW_CALLBACK = "%s.custom_show_pyinstrument" % __name__
157
158
159 You can configure the profile output type using setting's variable
160 PYINSTRUMENT_PROFILE_DIR_RENDERER. Default value is pyinstrument.ren‐
161 derers.HTMLRenderer. The supported renderers are pyinstrument.render‐
162 ers.JSONRenderer, pyinstrument.renderers.HTMLRenderer, pyinstru‐
163 ment.renderers.SpeedscopeRenderer.
164
165 Profile a web request in Flask
166 A simple setup to profile a Flask application is the following:
167
168 from flask import Flask, g, make_response, request
169 app = Flask(__name__)
170
171 @app.before_request
172 def before_request():
173 if "profile" in request.args:
174 g.profiler = Profiler()
175 g.profiler.start()
176
177
178 @app.after_request
179 def after_request(response):
180 if not hasattr(g, "profiler"):
181 return response
182 g.profiler.stop()
183 output_html = g.profiler.output_html()
184 return make_response(output_html)
185
186
187 This will check for the ?profile query param on each request and if
188 found, it starts profiling. After each request where the profiler was
189 running it creates the html output and returns that instead of the ac‐
190 tual response.
191
192 Profile a web request in FastAPI
193 To profile call stacks in FastAPI, you can write a middleware extension
194 for pyinstrument.
195
196 Create an async function and decorate with app.middleware('http') where
197 app is the name of your FastAPI application instance.
198
199 Make sure you configure a setting to only make this available when re‐
200 quired.
201
202 from pyinstrument import Profiler
203
204
205 PROFILING = True # Set this from a settings model
206
207 if PROFILING:
208 @app.middleware("http")
209 async def profile_request(request: Request, call_next):
210 profiling = request.query_params.get("profile", False)
211 if profiling:
212 profiler = Profiler(interval=settings.profiling_interval, async_mode="enabled")
213 profiler.start()
214 await call_next(request)
215 profiler.stop()
216 return HTMLResponse(profiler.output_html())
217 else:
218 return await call_next(request)
219
220
221 To invoke, make any request to your application with the GET parameter
222 profile=1 and it will print the HTML result from pyinstrument.
223
224 Profile Pytest tests
225 Pyinstrument can be invoked via the command-line to run pytest, giving
226 you a consolidated report for the test suite.
227
228 pyinstrument -m pytest [pytest-args...]
229
230
231 Or, to instrument specific tests, create and auto-use fixture in con‐
232 ftest.py in your test folder:
233
234 from pathlib import Path
235 import pytest
236 from pyinstrument import Profiler
237
238 TESTS_ROOT = Path.cwd()
239
240 @pytest.fixture(autouse=True)
241 def auto_profile(request):
242 PROFILE_ROOT = (TESTS_ROOT / ".profiles")
243 # Turn profiling on
244 profiler = Profiler()
245 profiler.start()
246
247 yield # Run test
248
249 profiler.stop()
250 PROFILE_ROOT.mkdir(exist_ok=True)
251 results_file = PROFILE_ROOT / f"{request.node.name}.html"
252 with open(results_file, "w", encoding="utf-8") as f_html:
253 f_html.write(profiler.output_html())
254
255
256 This will generate a HTML file for each test node in your test suite
257 inside the .profiles directory.
258
259 Profile something else?
260 I'd love to have more ways to profile using Pyinstrument - e.g. other
261 web frameworks. PRs are encouraged!
262
264 Pyinstrument interrupts the program every 1ms[1] and records the entire
265 stack at that point. It does this using a C extension and PyEval_Set‐
266 Profile, but only taking readings every 1ms. Check out this blog post
267 for more info.
268
269 You might be surprised at how few samples make up a report, but don't
270 worry, it won't decrease accuracy. The default interval of 1ms is a
271 lower bound for recording a stackframe, but if there is a long time
272 spent in a single function call, it will be recorded at the end of that
273 call. So effectively those samples were 'bunched up' and recorded at
274 the end.
275
276 Statistical profiling (not tracing)
277 Pyinstrument is a statistical profiler - it doesn't track every func‐
278 tion call that your program makes. Instead, it's recording the call
279 stack every 1ms.
280
281 That gives some advantages over other profilers. Firstly, statistical
282 profilers are much lower-overhead than tracing profilers.
283
284 ┌─────────────┬─────────────────────────────────────────┬──────────┐
285 │ │ Django template │ Overhead │
286 │ │ render × 4000 │ │
287 ├─────────────┼─────────────────────────────────────────┼──────────┤
288 │Base │ ████████████████ │ │
289 │ │ 0.33s │ │
290 ├─────────────┼─────────────────────────────────────────┼──────────┤
291 │ │ │ │
292 ├─────────────┼─────────────────────────────────────────┼──────────┤
293 │pyinstrument │ ████████████████████ │ 30% │
294 │ │ 0.43s │ │
295 ├─────────────┼─────────────────────────────────────────┼──────────┤
296 │cProfile │ █████████████████████████████ │ 84% │
297 │ │ 0.61s │ │
298 ├─────────────┼─────────────────────────────────────────┼──────────┤
299 │profile │ ██████████████████████████████████...██ │ 2057% │
300 │ │ 6.79s │ │
301 └─────────────┴─────────────────────────────────────────┴──────────┘
302
303 But low overhead is also important because it can distort the results.
304 When using a tracing profiler, code that makes a lot of Python function
305 calls invokes the profiler a lot, making it slower. This distorts the
306 results, and might lead you to optimise the wrong part of your program!
307
308 Full-stack recording
309 The standard Python profilers profile and cProfile show you a big list
310 of functions, ordered by the time spent in each function. This is
311 great, but it can be difficult to interpret why those functions are
312 getting called. It's more helpful to know why those functions are
313 called, and which parts of user code were involved.
314
315 For example, let's say I want to figure out why a web request in Django
316 is slow. If I use cProfile, I might get this:
317
318 151940 function calls (147672 primitive calls) in 1.696 seconds
319
320 Ordered by: cumulative time
321
322 ncalls tottime percall cumtime percall filename:lineno(function)
323 1 0.000 0.000 1.696 1.696 profile:0(<code object <module> at 0x1053d6a30, file "./manage.py", line 2>)
324 1 0.001 0.001 1.693 1.693 manage.py:2(<module>)
325 1 0.000 0.000 1.586 1.586 __init__.py:394(execute_from_command_line)
326 1 0.000 0.000 1.586 1.586 __init__.py:350(execute)
327 1 0.000 0.000 1.142 1.142 __init__.py:254(fetch_command)
328 43 0.013 0.000 1.124 0.026 __init__.py:1(<module>)
329 388 0.008 0.000 1.062 0.003 re.py:226(_compile)
330 158 0.005 0.000 1.048 0.007 sre_compile.py:496(compile)
331 1 0.001 0.001 1.042 1.042 __init__.py:78(get_commands)
332 153 0.001 0.000 1.036 0.007 re.py:188(compile)
333 106/102 0.001 0.000 1.030 0.010 __init__.py:52(__getattr__)
334 1 0.000 0.000 1.029 1.029 __init__.py:31(_setup)
335 1 0.000 0.000 1.021 1.021 __init__.py:57(_configure_logging)
336 2 0.002 0.001 1.011 0.505 log.py:1(<module>)
337
338
339 It's often hard to understand how your own code relates to these
340 traces.
341
342 Pyinstrument records the entire stack, so tracking expensive calls is
343 much easier. It also hides library frames by default, letting you focus
344 on your app/module is affecting performance.
345
346 _ ._ __/__ _ _ _ _ _/_ Recorded: 14:53:35 Samples: 131
347 /_//_/// /_\ / //_// / //_'/ // Duration: 3.131 CPU time: 0.195
348 / _/ v3.0.0b3
349
350 Program: examples/django_example/manage.py runserver --nothreading --noreload
351
352 3.131 <module> manage.py:2
353 └─ 3.118 execute_from_command_line django/core/management/__init__.py:378
354 [473 frames hidden] django, socketserver, selectors, wsgi...
355 2.836 select selectors.py:365
356 0.126 _get_response django/core/handlers/base.py:96
357 └─ 0.126 hello_world django_example/views.py:4
358
359
360 'Wall-clock' time (not CPU time)
361 Pyinstrument records duration using 'wall-clock' time. When you're
362 writing a program that downloads data, reads files, and talks to data‐
363 bases, all that time is included in the tracked time by pyinstrument.
364
365 That's really important when debugging performance problems, since
366 Python is often used as a 'glue' language between other services. The
367 problem might not be in your program, but you should still be able to
368 find why it's slow.
369
370 Async profiling
371 pyinstrument can profile async programs that use async and await. This
372 async support works by tracking the 'context' of execution, as provided
373 by the built-in contextvars module.
374
375 When you start a Profiler with the async_mode enabled or strict (not
376 disabled), that Profiler is attached to the current async context.
377
378 When profiling, pyinstrument keeps an eye on the context. When execu‐
379 tion exits the context, it captures the await stack that caused the
380 context to exit. Any time spent outside the context is attributed to
381 the that halted execution of the await.
382
383 Async contexts are inherited, so tasks started when a profiler is ac‐
384 tive are also profiled.
385
386 [image: Async context inheritance] [image]
387
388
389 pyinstrument supports async mode with Asyncio and Trio, other
390 async/await frameworks should work as long as they use contextvars.
391
392 Greenlet doesn't use async and await, and alters the Python stack dur‐
393 ing execution, so is not fully supported. However, because greenlet
394 also supports contextvars, we can limit profiling to one green thread,
395 using strict mode. In strict mode, whenever your green thread is halted
396 the time will be tracked in an <out-of-context> frame. Alternatively,
397 if you want to see what's happening when your green thread is halted,
398 you can use async_mode='disabled' - just be aware that readouts might
399 be misleading if multiple tasks are running concurrently.
400
401
402 ----
403
404
405
406 [1] Or, your configured interval.
407
409 Command line interface
410 pyinstrument works just like python, on the command line, so you can
411 call your scripts like pyinstrument script.py or pyinstrument -m
412 my_module.
413
414 When your script ends, or when you kill it with ctrl-c, pyinstrument
415 will print a profile report to the console.
416
417 System Message: ERROR/6 (/builddir/build/BUILD/pyinstru‐
418 ment-4.4.0/docs/reference.md:, line 12)
419 Command ['pyinstrument', '--help'] failed: [Errno 2] No such
420 file or directory: 'pyinstrument'
421
422 Python API
423 The Python API is also available, for calling pyinstrument directly
424 from Python and writing integrations with with other tools.
425
426 The Profiler object
427 class pyinstrument.Profiler(interval=0.001, async_mode='enabled')
428 The profiler - this is the main way to use pyinstrument.
429
430 Note the profiling will not start until start() is called.
431
432 Parameters
433
434 • interval (float) -- See interval.
435
436 • async_mode (AsyncMode) -- See async_mode.
437
438 property interval: float
439 The minimum time, in seconds, between each stack sample.
440 This translates into the resolution of the sampling.
441
442 property async_mode: str
443 Configures how this Profiler tracks time in a program
444 that uses async/await.
445
446 enabled
447 When this profiler sees an await, time is logged
448 in the function that awaited, rather than observ‐
449 ing other coroutines or the event loop.
450
451 disabled
452 This profiler doesn't attempt to track await. In a
453 program that uses async/await, this will inter‐
454 leave other coroutines and event loop machinery in
455 the profile. Use this option if async support is
456 causing issues in your use case, or if you want to
457 run multiple profilers at once.
458
459 strict Instructs the profiler to only profile the current
460 async context. Frames that are observed in an
461 other context are ignored, tracked instead as
462 <out-of-context>.
463
464 property last_session: pyinstrument.session.Session | None
465 The previous session recorded by the Profiler.
466
467 start(caller_frame=None)
468 Instructs the profiler to start - to begin observing the
469 program's execution and recording frames.
470
471 The normal way to invoke start() is with a new instance,
472 but you can restart a Profiler that was previously run‐
473 ning, too. The sessions are combined.
474
475 Parameters
476 caller_frame (frame | None) --
477
478 Set this to override the default behaviour of
479 treating the caller of start() as the
480 'start_call_stack' - the instigator of the pro‐
481 file. Most renderers will trim the 'root' from the
482 call stack up to this frame, to present a simpler
483 output.
484
485 You might want to set this to inspect.current‐
486 frame().f_back if you are writing a library that
487 wraps pyinstrument.
488
489
490 stop() Stops the profiler observing, and sets last_session to
491 the captured session.
492
493 Returns
494 The captured session.
495
496 Return type
497 Session
498
499 property is_running
500 Returns True if this profiler is running - i.e. observing
501 the program execution.
502
503 reset()
504 Resets the Profiler, clearing the last_session.
505
506 __enter__()
507 Context manager support.
508
509 Profilers can be used in with blocks! See this example:
510
511 with Profiler() as p:
512 # your code here...
513 do_some_work()
514
515 # profiling has ended. let's print the output.
516 p.print()
517
518 print(file=sys.stdout, *, unicode=None, color=None,
519 show_all=False, timeline=False)
520 Print the captured profile to the console.
521
522 Parameters
523
524 • file (IO[str]) -- the IO stream to write to.
525 Could be a file descriptor or sys.stdout,
526 sys.stderr. Defaults to sys.stdout.
527
528 • unicode (bool | None) -- Override unicode sup‐
529 port detection.
530
531 • color (bool | None) -- Override ANSI color sup‐
532 port detection.
533
534 • show_all (bool) -- Sets the show_all parameter
535 on the renderer.
536
537 • timeline (bool) -- Sets the timeline parameter
538 on the renderer.
539
540 output_text(unicode=False, color=False, show_all=False, time‐
541 line=False)
542 Return the profile output as text, as rendered by Consol‐
543 eRenderer
544
545 output_html(timeline=False)
546 Return the profile output as HTML, as rendered by HTML‐
547 Renderer
548
549 open_in_browser(timeline=False)
550 Opens the last profile session in your web browser.
551
552 output(renderer)
553 Returns the last profile session, as rendered by ren‐
554 derer.
555
556 Parameters
557 renderer (Renderer) -- The renderer to use.
558
559 Sessions
560 class pyinstrument.session.Session
561 Represents a profile session, contains the data collected during
562 a profile session.
563
564 static load(filename)
565 Load a previously saved session from disk.
566
567 Parameters
568 filename (PathOrStr) -- The path to load from.
569
570 Return type
571 Session
572
573 save(filename)
574 Saves a Session object to disk, in a JSON format.
575
576 Parameters
577 filename (PathOrStr) -- The path to save to. Using
578 the .pyisession extension is recommended.
579
580 static combine(session1, session2)
581 Combines two Session objects.
582
583 Sessions that are joined in this way probably shouldn't
584 be interpreted as timelines, because the samples are sim‐
585 ply concatenated. But aggregate views (the default) of
586 this data will work.
587
588 Return type
589 Session
590
591 root_frame(trim_stem=True)
592 Parses the internal frame records and returns a tree of
593 Frame objects. This object can be renderered using a Ren‐
594 derer object.
595
596 Return type
597 A Frame object, or None if the session is empty.
598
599 Renderers
600 Renderers transform a tree of Frame objects into some form of output.
601
602 Rendering has two steps:
603
604 1. First, the renderer will 'preprocess' the Frame tree, applying each
605 processor in the processor property, in turn.
606
607 2. The resulting tree is renderered into the desired format.
608
609 Therefore, rendering can be customised by changing the processors prop‐
610 erty. For example, you can disable time-aggregation (making the profile
611 into a timeline) by removing aggregate_repeated_calls().
612
613 class pyinstrument.renderers.FrameRenderer(show_all=False, time‐
614 line=False, processor_options=None)
615 An abstract base class for renderers that process Frame objects
616 using processor functions. Provides a common interface to manip‐
617 ulate the processors before rendering.
618
619 Parameters
620
621 • show_all (bool) -- Don't hide library frames - show ev‐
622 erything that pyinstrument captures.
623
624 • timeline (bool) -- Instead of aggregating time, leave
625 the samples in chronological order.
626
627 • processor_options (dict[str, Any]) -- A dictionary of
628 processor options.
629
630 processors: List[Callable[[...], Optional[Frame]]]
631 Processors installed on this renderer. This property is
632 defined on the base class to provide a common way for
633 users to add and manipulate them before calling render().
634
635 processor_options: dict[str, Any]
636 Dictionary containing processor options, passed to each
637 processor.
638
639 default_processors()
640 Return a list of processors that this renderer uses by
641 default.
642
643 render(session)
644 Return a string that contains the rendered form of frame.
645
646 class pyinstrument.renderers.ConsoleRenderer(unicode=False,
647 color=False, time='seconds', **kwargs)
648 Produces text-based output, suitable for text files or ANSI-com‐
649 patible consoles.
650
651 Parameters
652
653 • unicode (bool) -- Use unicode, like box-drawing charac‐
654 ters in the output.
655
656 • color (bool) -- Enable color support, using ANSI color
657 sequences.
658
659 • time (str) -- How to display the duration of each frame
660 - 'seconds' or 'percent_of_total'
661
662 class pyinstrument.renderers.HTMLRenderer(**kwargs)
663 Renders a rich, interactive web page, as a string of HTML.
664
665 Parameters
666
667 • show_all -- Don't hide library frames - show everything
668 that pyinstrument captures.
669
670 • timeline -- Instead of aggregating time, leave the sam‐
671 ples in chronological order.
672
673 • processor_options -- A dictionary of processor options.
674
675 class pyinstrument.renderers.JSONRenderer(**kwargs)
676 Outputs a tree of JSON, containing processed frames.
677
678 Parameters
679
680 • show_all -- Don't hide library frames - show everything
681 that pyinstrument captures.
682
683 • timeline -- Instead of aggregating time, leave the sam‐
684 ples in chronological order.
685
686 • processor_options -- A dictionary of processor options.
687
688 class pyinstrument.renderers.SpeedscopeRenderer(**kwargs)
689 Outputs a tree of JSON conforming to the speedscope schema docu‐
690 mented at
691
692 wiki:
693 https://github.com/jlfwong/speedscope/wiki/Importing-from-custom-sources
694 schema: https://www.speedscope.app/file-format-schema.json spec:
695 https://github.com/jlfwong/speedscope/blob/main/src/lib/file-format-spec.ts
696 example:
697 https://github.com/jlfwong/speedscope/blob/main/sample/profiles/speedscope/0.0.1/simple.speedscope.json
698
699 Parameters
700
701 • show_all -- Don't hide library frames - show everything
702 that pyinstrument captures.
703
704 • timeline -- Instead of aggregating time, leave the sam‐
705 ples in chronological order.
706
707 • processor_options -- A dictionary of processor options.
708
709 Processors
710 Processors are functions that take a Frame object, and mutate the tree
711 to perform some task.
712
713 They can mutate the tree in-place, but also can change the root frame,
714 they should always be called like:
715
716 frame = processor(frame, options=...)
717
718 pyinstrument.processors.remove_importlib(frame, options)
719 Removes <frozen importlib._bootstrap frames that clutter the
720 output.
721
722 pyinstrument.processors.remove_tracebackhide(frame, options)
723 Removes frames that have set a local __tracebackhide__ (e.g.
724 __tracebackhide__ = True), to hide them from the output.
725
726 pyinstrument.processors.aggregate_repeated_calls(frame, options)
727 Converts a timeline into a time-aggregate summary.
728
729 Adds together calls along the same call stack, so that repeated
730 calls appear as the same frame. Removes time-linearity - frames
731 are sorted according to total time spent.
732
733 Useful for outputs that display a summary of execution (e.g.
734 text and html outputs)
735
736 pyinstrument.processors.group_library_frames_processor(frame, options)
737 Groups frames that should be hidden into FrameGroup objects, ac‐
738 cording to hide_regex and show_regex in the options dict, as ap‐
739 plied to the file path of the source code of the frame. If both
740 match, 'show' has precedence. Options:
741
742 hide_regex
743 regular expression, which if matches the file path, hides
744 the frame in a frame group.
745
746 show_regex
747 regular expression, which if matches the file path, en‐
748 sures the frame is not hidden
749
750 Single frames are not grouped, there must be at least two frames
751 in a group.
752
753 pyinstrument.processors.merge_consecutive_self_time(frame, options, re‐
754 cursive=True)
755 Combines consecutive 'self time' frames.
756
757 pyinstrument.processors.remove_unnecessary_self_time_nodes(frame, op‐
758 tions)
759 When a frame has only one child, and that is a self-time frame,
760 remove that node and move the time to parent, since it's unnec‐
761 essary - it clutters the output and offers no additional infor‐
762 mation.
763
764 pyinstrument.processors.remove_irrelevant_nodes(frame, options, to‐
765 tal_time=None)
766 Remove nodes that represent less than e.g. 1% of the output. Op‐
767 tions:
768
769 filter_threshold
770 sets the minimum duration of a frame to be included in
771 the output. Default: 0.01.
772
773 pyinstrument.processors.remove_first_pyinstrument_frames_proces‐
774 sor(frame, options)
775 The first few frames when using the command line are the
776 __main__ of pyinstrument, the eval, and the 'runpy' module. I
777 want to remove that from the output.
778
779 Internals notes
780 Frames are recorded by the Profiler in a time-linear fashion. While
781 profiling, the profiler builds a list of frame stacks, with the frames
782 having in format:
783
784 function_name <null> filename <null> function_line_number
785
786
787 When profiling is complete, this list is turned into a tree structure
788 of Frame objects. This tree contains all the information as gathered by
789 the profiler, suitable for a flame render.
790
791 Frame objects, the call tree, and processors
792 The frames are assembled to a call tree by the profiler session. The
793 time-linearity is retained at this stage.
794
795 Before rendering, the call tree is then fed through a sequence of 'pro‐
796 cessors' to transform the tree for output.
797
798 The most interesting is aggregate_repeated_calls, which combines dif‐
799 ferent instances of function calls into the same frame. This is intu‐
800 itive as a summary of where time was spent during execution.
801
802 The rest of the processors focus on removing or hiding irrelevant
803 Frames from the output.
804
805 Self time frames vs. frame.self_time
806 Self time nodes exist to record time spent in a node, but not in its
807 children. But normal frame objects can have self_time too. Why?
808 frame.self_time is used to store the self_time of any nodes that were
809 removed during processing.
810
812 • Index
813
814 • Search Page
815
817 Joe Rickerby
818
820 2023, Joe Rickerby
821
822
823
824
825 May 15, 2023 PYINSTRUMENT(1)