{"id":408,"date":"2025-08-09T15:31:39","date_gmt":"2025-08-09T15:31:39","guid":{"rendered":"https:\/\/dataopsschool.com\/blog\/?p=408"},"modified":"2025-08-09T16:25:08","modified_gmt":"2025-08-09T16:25:08","slug":"databricks-lab-excercise-notebook","status":"publish","type":"post","link":"https:\/\/dataopsschool.com\/blog\/databricks-lab-excercise-notebook\/","title":{"rendered":"Databricks Lab &amp; Excercise &#8211; Notebook"},"content":{"rendered":"\n<p>Here\u2019s my <strong>Top 15 commands to try first<\/strong> \u2014 grouped into <strong>environment checks<\/strong>, <strong>Spark basics<\/strong>, and <strong>data handling<\/strong> so you learn in a logical order.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>1\u20135: Environment &amp; Python Basics in Databricks<\/strong><\/h2>\n\n\n\n<pre class=\"wp-block-code\"><code># 1. Check Python version\n!python --version\n\n# 2. List installed Python packages\n%pip list\n\n# 3. Display current working directory\n%fs pwd\n\n# 4. List files in DBFS root\n%fs ls \/\n\n# 5. Display help for Databricks magic commands\n%magic\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>6\u201310: Spark Session &amp; Cluster Basics<\/strong><\/h2>\n\n\n\n<pre class=\"wp-block-code\"><code># 6. Create a Spark session (already available by default in Databricks)\nspark\n\n# 7. Check Spark version\nspark.version\n\n# 8. View Spark configuration settings\nspark.conf.getAll()\n\n# 9. View list of databases\nspark.sql(\"SHOW DATABASES\").show()\n\n# 10. Check available tables in default database\nspark.sql(\"SHOW TABLES\").show()\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>11\u201315: Data Creation, Querying, and Display<\/strong><\/h2>\n\n\n\n<pre class=\"wp-block-code\"><code>from pyspark.sql import Row\n\n# 11. Create a simple DataFrame\ndata = &#91;Row(name=\"Alice\", age=29), Row(name=\"Bob\", age=35)]\ndf = spark.createDataFrame(data)\n\n# 12. Show DataFrame\ndf.show()\n\n# 13. Describe DataFrame schema\ndf.printSchema()\n\n# 14. Run SQL on the DataFrame\ndf.createOrReplaceTempView(\"people\")\nspark.sql(\"SELECT name, age FROM people WHERE age &gt; 30\").show()\n\n# 15. Save DataFrame as table in default DB\ndf.write.saveAsTable(\"people_table\")\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Bonus Tips for First Run<\/strong><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use <code>%python<\/code> explicitly if you\u2019re mixing languages in the same notebook.<\/li>\n\n\n\n<li><code>%fs<\/code> is for DBFS (Databricks File System) commands \u2014 good for browsing uploaded data.<\/li>\n\n\n\n<li><code>%sql<\/code> lets you run SQL directly in a cell without <code>spark.sql(...)<\/code>.<\/li>\n\n\n\n<li>Try <code>display(df)<\/code> \u2014 it gives a nice interactive table UI instead of plain text.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<pre class=\"wp-block-code\"><code>\nIPython's 'magic' functions\n===========================\n\nThe magic function system provides a series of functions which allow you to\ncontrol the behavior of IPython itself, plus a lot of system-type\nfeatures. There are two kinds of magics, line-oriented and cell-oriented.\n\nLine magics are prefixed with the % character and work much like OS\ncommand-line calls: they get as an argument the rest of the line, where\narguments are passed without parentheses or quotes.  For example, this will\ntime the given statement::\n\n        %timeit range(1000)\n\nCell magics are prefixed with a double %%, and they are functions that get as\nan argument not only the rest of the line, but also the lines below it in a\nseparate argument.  These magics are called with two arguments: the rest of the\ncall line and the body of the cell, consisting of the lines below the first.\nFor example::\n\n        %%timeit x = numpy.random.randn((100, 100))\n        numpy.linalg.svd(x)\n\nwill time the execution of the numpy svd routine, running the assignment of x\nas part of the setup phase, which is not timed.\n\nIn a line-oriented client (the terminal or Qt console IPython), starting a new\ninput with %% will automatically enter cell mode, and IPython will continue\nreading input until a blank line is given.  In the notebook, simply type the\nwhole cell as one entity, but keep in mind that the %% escape can only be at\nthe very start of the cell.\n\nNOTE: If you have 'automagic' enabled (via the command line option or with the\n%automagic function), you don't need to type in the % explicitly for line\nmagics; cell magics always require an explicit '%%' escape.  By default,\nIPython ships with automagic on, so you should only rarely need the % escape.\n\nExample: typing '%cd mydir' (without the quotes) changes your working directory\nto 'mydir', if it exists.\n\nFor a list of the available magic functions, use %lsmagic. For a description\nof any of them, type %magic_name?, e.g. '%cd?'.\n\nCurrently the magic system has the following functions:\n%alias:\n    Define an alias for a system command.\n    \n    '%alias alias_name cmd' defines 'alias_name' as an alias for 'cmd'\n    \n    Then, typing 'alias_name params' will execute the system command 'cmd\n    params' (from your underlying operating system).\n    \n    Aliases have lower precedence than magic functions and Python normal\n    variables, so if 'foo' is both a Python variable and an alias, the\n    alias can not be executed until 'del foo' removes the Python variable.\n    \n    You can use the %l specifier in an alias definition to represent the\n    whole line when the alias is called.  For example::\n    \n      In &#91;2]: alias bracket echo \"Input in brackets: &lt;%l>\"\n      In &#91;3]: bracket hello world\n      Input in brackets: &lt;hello world>\n    \n    You can also define aliases with parameters using %s specifiers (one\n    per parameter)::\n    \n      In &#91;1]: alias parts echo first %s second %s\n      In &#91;2]: %parts A B\n      first A second B\n      In &#91;3]: %parts A\n      Incorrect number of arguments: 2 expected.\n      parts is an alias to: 'echo first %s second %s'\n    \n    Note that %l and %s are mutually exclusive.  You can only use one or\n    the other in your aliases.\n    \n    Aliases expand Python variables just like system calls using ! or !!\n    do: all expressions prefixed with '$' get expanded.  For details of\n    the semantic rules, see PEP-215:\n    https:\/\/peps.python.org\/pep-0215\/.  This is the library used by\n    IPython for variable expansion.  If you want to access a true shell\n    variable, an extra $ is necessary to prevent its expansion by\n    IPython::\n    \n      In &#91;6]: alias show echo\n      In &#91;7]: PATH='A Python string'\n      In &#91;8]: show $PATH\n      A Python string\n      In &#91;9]: show $$PATH\n      \/usr\/local\/lf9560\/bin:\/usr\/local\/intel\/compiler70\/ia32\/bin:...\n    \n    You can use the alias facility to access all of $PATH.  See the %rehashx\n    function, which automatically creates aliases for the contents of your\n    $PATH.\n    \n    If called with no parameters, %alias prints the current alias table\n    for your system.  For posix systems, the default aliases are 'cat',\n    'cp', 'mv', 'rm', 'rmdir', and 'mkdir', and other platform-specific\n    aliases are added.  For windows-based systems, the default aliases are\n    'copy', 'ddir', 'echo', 'ls', 'ldir', 'mkdir', 'ren', and 'rmdir'.\n    \n    You can see the definition of alias by adding a question mark in the\n    end::\n    \n      In &#91;1]: cat?\n      Repr: &lt;alias cat for 'cat'>\n%alias_magic:\n    ::\n    \n      %alias_magic &#91;-l] &#91;-c] &#91;-p PARAMS] name target\n    \n    Create an alias for an existing line or cell magic.\n    \n    Examples\n    --------\n    ::\n    \n      In &#91;1]: %alias_magic t timeit\n      Created `%t` as an alias for `%timeit`.\n      Created `%%t` as an alias for `%%timeit`.\n    \n      In &#91;2]: %t -n1 pass\n      107 ns \u00b1 43.6 ns per loop (mean \u00b1 std. dev. of 7 runs, 1 loop each)\n    \n      In &#91;3]: %%t -n1\n         ...: pass\n         ...:\n      107 ns \u00b1 58.3 ns per loop (mean \u00b1 std. dev. of 7 runs, 1 loop each)\n    \n      In &#91;4]: %alias_magic --cell whereami pwd\n      UsageError: Cell magic function `%%pwd` not found.\n      In &#91;5]: %alias_magic --line whereami pwd\n      Created `%whereami` as an alias for `%pwd`.\n    \n      In &#91;6]: %whereami\n      Out&#91;6]: '\/home\/testuser'\n    \n      In &#91;7]: %alias_magic h history -p \"-l 30\" --line\n      Created `%h` as an alias for `%history -l 30`.\n    \n    positional arguments:\n      name                  Name of the magic to be created.\n      target                Name of the existing line or cell magic.\n    \n    options:\n      -l, --line            Create a line magic alias.\n      -c, --cell            Create a cell magic alias.\n      -p PARAMS, --params PARAMS\n                            Parameters passed to the magic function.\n%autoawait:\n    \n    Allow to change the status of the autoawait option.\n    \n    This allow you to set a specific asynchronous code runner.\n    \n    If no value is passed, print the currently used asynchronous integration\n    and whether it is activated.\n    \n    It can take a number of value evaluated in the following order:\n    \n    - False\/false\/off deactivate autoawait integration\n    - True\/true\/on activate autoawait integration using configured default\n      loop\n    - asyncio\/curio\/trio activate autoawait integration and use integration\n      with said library.\n    \n    - `sync` turn on the pseudo-sync integration (mostly used for\n      `IPython.embed()` which does not run IPython with a real eventloop and\n      deactivate running asynchronous code. Turning on Asynchronous code with\n      the pseudo sync loop is undefined behavior and may lead IPython to crash.\n    \n    If the passed parameter does not match any of the above and is a python\n    identifier, get said object from user namespace and set it as the\n    runner, and activate autoawait.\n    \n    If the object is a fully qualified object name, attempt to import it and\n    set it as the runner, and activate autoawait.\n    \n    The exact behavior of autoawait is experimental and subject to change\n    across version of IPython and Python.\n%autocall:\n    Make functions callable without having to type parentheses.\n    \n    Usage:\n    \n       %autocall &#91;mode]\n    \n    The mode can be one of: 0->Off, 1->Smart, 2->Full.  If not given, the\n    value is toggled on and off (remembering the previous state).\n    \n    In more detail, these values mean:\n    \n    0 -> fully disabled\n    \n    1 -> active, but do not apply if there are no arguments on the line.\n    \n    In this mode, you get::\n    \n      In &#91;1]: callable\n      Out&#91;1]: &lt;built-in function callable>\n    \n      In &#91;2]: callable 'hello'\n      ------> callable('hello')\n      Out&#91;2]: False\n    \n    2 -> Active always.  Even if no arguments are present, the callable\n    object is called::\n    \n      In &#91;2]: float\n      ------> float()\n      Out&#91;2]: 0.0\n    \n    Note that even with autocall off, you can still use '\/' at the start of\n    a line to treat the first argument on the command line as a function\n    and add parentheses to it::\n    \n      In &#91;8]: \/str 43\n      ------> str(43)\n      Out&#91;8]: '43'\n    \n    # all-random (note for auto-testing)\n%automagic:\n    Make magic functions callable without having to type the initial %.\n    \n    Without arguments toggles on\/off (when off, you must call it as\n    %automagic, of course).  With arguments it sets the value, and you can\n    use any of (case insensitive):\n    \n     - on, 1, True: to activate\n    \n     - off, 0, False: to deactivate.\n    \n    Note that magic functions have lowest priority, so if there's a\n    variable whose name collides with that of a magic fn, automagic won't\n    work for that function (you get the variable instead). However, if you\n    delete the variable (del var), the previously shadowed magic function\n    becomes visible to automagic again.\n%autosave:\n    Set the autosave interval in the notebook (in seconds).\n    \n    The default value is 120, or two minutes.\n    ``%autosave 0`` will disable autosave.\n    \n    This magic only has an effect when called from the notebook interface.\n    It has no effect when called in a startup file.\n%bookmark:\n    Manage IPython's bookmark system.\n    \n    %bookmark &lt;name>       - set bookmark to current dir\n    %bookmark &lt;name> &lt;dir> - set bookmark to &lt;dir>\n    %bookmark -l           - list all bookmarks\n    %bookmark -d &lt;name>    - remove bookmark\n    %bookmark -r           - remove all bookmarks\n    \n    You can later on access a bookmarked folder with::\n    \n      %cd -b &lt;name>\n    \n    or simply '%cd &lt;name>' if there is no directory called &lt;name> AND\n    there is such a bookmark defined.\n    \n    Your bookmarks persist through IPython sessions, but they are\n    associated with each profile.\n%cat:\n    Alias for `!cat`\n%cd:\n    Change the current working directory.\n    \n    This command automatically maintains an internal list of directories\n    you visit during your IPython session, in the variable ``_dh``. The\n    command :magic:`%dhist` shows this history nicely formatted. You can\n    also do ``cd -&lt;tab>`` to see directory history conveniently.\n    Usage:\n    \n      - ``cd 'dir'``: changes to directory 'dir'.\n      - ``cd -``: changes to the last visited directory.\n      - ``cd -&lt;n>``: changes to the n-th directory in the directory history.\n      - ``cd --foo``: change to directory that matches 'foo' in history\n      - ``cd -b &lt;bookmark_name>``: jump to a bookmark set by %bookmark\n      - Hitting a tab key after ``cd -b`` allows you to tab-complete\n        bookmark names.\n    \n      .. note::\n        ``cd &lt;bookmark_name>`` is enough if there is no directory\n        ``&lt;bookmark_name>``, but a bookmark with the name exists.\n    \n    Options:\n    \n    -q               Be quiet. Do not print the working directory after the\n                     cd command is executed. By default IPython's cd\n                     command does print this directory, since the default\n                     prompts do not display path information.\n    \n    .. note::\n       Note that ``!cd`` doesn't work for this purpose because the shell\n       where ``!command`` runs is immediately discarded after executing\n       'command'.\n    \n    Examples\n    --------\n    ::\n    \n      In &#91;10]: cd parent\/child\n      \/home\/tsuser\/parent\/child\n%clear:\n    Clear the terminal.\n%code_wrap:\n    ::\n    \n      %code_wrap &#91;--remove] &#91;--list] &#91;--list-all] &#91;name]\n    \n    Simple magic to quickly define a code transformer for all IPython's future input.\n    \n    ``__code__`` and ``__ret__`` are special variable that represent the code to run\n    and the value of the last expression of ``__code__`` respectively.\n    \n    Examples\n    --------\n    \n    .. ipython::\n    \n        In &#91;1]: %%code_wrap before_after\n           ...: print('before')\n           ...: __code__\n           ...: print('after')\n           ...: __ret__\n    \n        In &#91;2]: 1\n        before\n        after\n        Out&#91;2]: 1\n    \n        In &#91;3]: %code_wrap --list\n        before_after\n    \n        In &#91;4]: %code_wrap --list-all\n        before_after :\n            print('before')\n            __code__\n            print('after')\n            __ret__\n    \n        In &#91;5]: %code_wrap --remove before_after\n    \n    positional arguments:\n      name\n    \n    options:\n      --remove    remove the current transformer\n      --list      list existing transformers name\n      --list-all  list existing transformers name and code template\n%colors:\n    Switch color scheme for prompts, info system and exception handlers.\n    \n    Currently implemented schemes: NoColor, Linux, LightBG.\n    \n    Color scheme names are not case-sensitive.\n    \n    Examples\n    --------\n    To get a plain black and white terminal::\n    \n      %colors nocolor\n%conda:\n    No documentation\n%config:\n    configure IPython\n    \n        %config Class&#91;.trait=value]\n    \n    This magic exposes most of the IPython config system. Any\n    Configurable class should be able to be configured with the simple\n    line::\n    \n        %config Class.trait=value\n    \n    Where `value` will be resolved in the user's namespace, if it is an\n    expression or variable name.\n    \n    Examples\n    --------\n    \n    To see what classes are available for config, pass no arguments::\n    \n        In &#91;1]: %config\n        Available objects for config:\n            AliasManager\n            DisplayFormatter\n            HistoryManager\n            IPCompleter\n            LoggingMagics\n            MagicsManager\n            OSMagics\n            PrefilterManager\n            ScriptMagics\n            TerminalInteractiveShell\n    \n    To view what is configurable on a given class, just pass the class\n    name::\n    \n        In &#91;2]: %config LoggingMagics\n        LoggingMagics(Magics) options\n        ---------------------------\n        LoggingMagics.quiet=&lt;Bool>\n            Suppress output of log state when logging is enabled\n            Current: False\n    \n    but the real use is in setting values::\n    \n        In &#91;3]: %config LoggingMagics.quiet = True\n    \n    and these values are read from the user_ns if they are variables::\n    \n        In &#91;4]: feeling_quiet=False\n    \n        In &#91;5]: %config LoggingMagics.quiet = feeling_quiet\n%connect_info:\n    Print information for connecting other clients to this kernel\n    \n    It will print the contents of this session's connection file, as well as\n    shortcuts for local clients.\n    \n    In the simplest case, when called from the most recently launched kernel,\n    secondary clients can be connected, simply with:\n    \n    $> jupyter &lt;app> --existing\n%cp:\n    Alias for `!cp`\n%debug:\n    ::\n    \n      %debug &#91;--breakpoint FILE:LINE] &#91;statement ...]\n    \n    Activate the interactive debugger.\n    \n    This magic command support two ways of activating debugger.\n    One is to activate debugger before executing code.  This way, you\n    can set a break point, to step through the code from the point.\n    You can use this mode by giving statements to execute and optionally\n    a breakpoint.\n    \n    The other one is to activate debugger in post-mortem mode.  You can\n    activate this mode simply running %debug without any argument.\n    If an exception has just occurred, this lets you inspect its stack\n    frames interactively.  Note that this will always work only on the last\n    traceback that occurred, so you must call this quickly after an\n    exception that you wish to inspect has fired, because if another one\n    occurs, it clobbers the previous one.\n    \n    If you want IPython to automatically do this on every exception, see\n    the %pdb magic for more details.\n    \n    .. versionchanged:: 7.3\n        When running code, user variables are no longer expanded,\n        the magic line is always left unmodified.\n    \n    positional arguments:\n      statement             Code to run in debugger. You can omit this in cell\n                            magic mode.\n    \n    options:\n      --breakpoint &lt;FILE:LINE>, -b &lt;FILE:LINE>\n                            Set break point at LINE in FILE.\n%dhist:\n    Print your history of visited directories.\n    \n    %dhist       -> print full history\n    %dhist n     -> print last n entries only\n    %dhist n1 n2 -> print entries between n1 and n2 (n2 not included)\n    \n    This history is automatically maintained by the %cd command, and\n    always available as the global list variable _dh. You can use %cd -&lt;n>\n    to go to directory number &lt;n>.\n    \n    Note that most of time, you should view directory history by entering\n    cd -&lt;TAB>.\n%dirs:\n    Return the current directory stack.\n%doctest_mode:\n    Toggle doctest mode on and off.\n    \n    This mode is intended to make IPython behave as much as possible like a\n    plain Python shell, from the perspective of how its prompts, exceptions\n    and output look.  This makes it easy to copy and paste parts of a\n    session into doctests.  It does so by:\n    \n    - Changing the prompts to the classic ``>>>`` ones.\n    - Changing the exception reporting mode to 'Plain'.\n    - Disabling pretty-printing of output.\n    \n    Note that IPython also supports the pasting of code snippets that have\n    leading '>>>' and '...' prompts in them.  This means that you can paste\n    doctests from files or docstrings (even if they have leading\n    whitespace), and the code will execute correctly.  You can then use\n    '%history -t' to see the translated history; this will give you the\n    input after removal of all the leading prompts and whitespace, which\n    can be pasted back into an editor.\n    \n    With these features, you can switch into this mode easily whenever you\n    need to do testing and changes to doctests, without having to leave\n    your existing IPython session.\n%ed:\n    Alias for `%edit`.\n%edit:\n    Bring up an editor and execute the resulting code.\n    \n    Usage:\n      %edit &#91;options] &#91;args]\n    \n    %edit runs an external text editor. You will need to set the command for\n    this editor via the ``TerminalInteractiveShell.editor`` option in your\n    configuration file before it will work.\n    \n    This command allows you to conveniently edit multi-line code right in\n    your IPython session.\n    \n    If called without arguments, %edit opens up an empty editor with a\n    temporary file and will execute the contents of this file when you\n    close it (don't forget to save it!).\n    \n    Options:\n    \n    -n &lt;number>\n      Open the editor at a specified line number. By default, the IPython\n      editor hook uses the unix syntax 'editor +N filename', but you can\n      configure this by providing your own modified hook if your favorite\n      editor supports line-number specifications with a different syntax.\n    \n    -p\n      Call the editor with the same data as the previous time it was used,\n      regardless of how long ago (in your current session) it was.\n    \n    -r\n      Use 'raw' input. This option only applies to input taken from the\n      user's history.  By default, the 'processed' history is used, so that\n      magics are loaded in their transformed version to valid Python.  If\n      this option is given, the raw input as typed as the command line is\n      used instead.  When you exit the editor, it will be executed by\n      IPython's own processor.\n    \n    Arguments:\n    \n    If arguments are given, the following possibilities exist:\n    \n    - The arguments are numbers or pairs of colon-separated numbers (like\n      1 4:8 9). These are interpreted as lines of previous input to be\n      loaded into the editor. The syntax is the same of the %macro command.\n    \n    - If the argument doesn't start with a number, it is evaluated as a\n      variable and its contents loaded into the editor. You can thus edit\n      any string which contains python code (including the result of\n      previous edits).\n    \n    - If the argument is the name of an object (other than a string),\n      IPython will try to locate the file where it was defined and open the\n      editor at the point where it is defined. You can use ``%edit function``\n      to load an editor exactly at the point where 'function' is defined,\n      edit it and have the file be executed automatically.\n    \n      If the object is a macro (see %macro for details), this opens up your\n      specified editor with a temporary file containing the macro's data.\n      Upon exit, the macro is reloaded with the contents of the file.\n    \n      Note: opening at an exact line is only supported under Unix, and some\n      editors (like kedit and gedit up to Gnome 2.8) do not understand the\n      '+NUMBER' parameter necessary for this feature. Good editors like\n      (X)Emacs, vi, jed, pico and joe all do.\n    \n    - If the argument is not found as a variable, IPython will look for a\n      file with that name (adding .py if necessary) and load it into the\n      editor. It will execute its contents with execfile() when you exit,\n      loading any code in the file into your interactive namespace.\n    \n    Unlike in the terminal, this is designed to use a GUI editor, and we do\n    not know when it has closed. So the file you edit will not be\n    automatically executed or printed.\n    \n    Note that %edit is also available through the alias %ed.\n%env:\n    Get, set, or list environment variables.\n    \n    Usage:\n    \n      :``%env``: lists all environment variables\/values\n      :``%env var``: get value for var\n      :``%env var val``: set value for var\n      :``%env var=val``: set value for var\n      :``%env var=$val``: set value for var, using python expansion if possible\n%gui:\n    Enable or disable IPython GUI event loop integration.\n    \n    %gui &#91;GUINAME]\n    \n    This magic replaces IPython's threaded shells that were activated\n    using the (pylab\/wthread\/etc.) command line flags.  GUI toolkits\n    can now be enabled at runtime and keyboard\n    interrupts should work without any problems.  The following toolkits\n    are supported:  wxPython, PyQt4, PyGTK, Tk and Cocoa (OSX)::\n    \n        %gui wx      # enable wxPython event loop integration\n        %gui qt      # enable PyQt\/PySide event loop integration\n                     # with the latest version available.\n        %gui qt6     # enable PyQt6\/PySide6 event loop integration\n        %gui qt5     # enable PyQt5\/PySide2 event loop integration\n        %gui gtk     # enable PyGTK event loop integration\n        %gui gtk3    # enable Gtk3 event loop integration\n        %gui gtk4    # enable Gtk4 event loop integration\n        %gui tk      # enable Tk event loop integration\n        %gui osx     # enable Cocoa event loop integration\n                     # (requires %matplotlib 1.1)\n        %gui         # disable all event loop integration\n    \n    WARNING:  after any of these has been called you can simply create\n    an application object, but DO NOT start the event loop yourself, as\n    we have already handled that.\n%hist:\n    Alias for `%history`.\n%history:\n    ::\n    \n      %history &#91;-n] &#91;-o] &#91;-p] &#91;-t] &#91;-f FILENAME] &#91;-g &#91;PATTERN ...]]\n                   &#91;-l &#91;LIMIT]] &#91;-u]\n                   &#91;range ...]\n    \n    Print input history (_i&lt;n> variables), with most recent last.\n    \n    By default, input history is printed without line numbers so it can be\n    directly pasted into an editor. Use -n to show them.\n    \n    By default, all input history from the current session is displayed.\n    Ranges of history can be indicated using the syntax:\n    \n    ``4``\n        Line 4, current session\n    ``4-6``\n        Lines 4-6, current session\n    ``243\/1-5``\n        Lines 1-5, session 243\n    ``~2\/7``\n        Line 7, session 2 before current\n    ``~8\/1-~6\/5``\n        From the first line of 8 sessions ago, to the fifth line of 6\n        sessions ago.\n    \n    Multiple ranges can be entered, separated by spaces\n    \n    The same syntax is used by %macro, %save, %edit, %rerun\n    \n    Examples\n    --------\n    ::\n    \n      In &#91;6]: %history -n 4-6\n      4:a = 12\n      5:print(a**2)\n      6:%history -n 4-6\n    \n    positional arguments:\n      range\n    \n    options:\n      -n                  print line numbers for each input. This feature is only\n                          available if numbered prompts are in use.\n      -o                  also print outputs for each input.\n      -p                  print classic '>>>' python prompts before each input.\n                          This is useful for making documentation, and in\n                          conjunction with -o, for producing doctest-ready output.\n      -t                  print the 'translated' history, as IPython understands\n                          it. IPython filters your input and converts it all into\n                          valid Python source before executing it (things like\n                          magics or aliases are turned into function calls, for\n                          example). With this option, you'll see the native\n                          history instead of the user-entered version: '%cd \/'\n\n\n*** WARNING: max output size exceeded, skipping output. ***\n\nension`` function.\n%who:\n    Print all interactive variables, with some minimal formatting.\n    \n    If any arguments are given, only variables whose type matches one of\n    these are printed.  For example::\n    \n      %who function str\n    \n    will only list functions and strings, excluding all other types of\n    variables.  To find the proper type names, simply use type(var) at a\n    command line to see how python prints type names.  For example:\n    \n    ::\n    \n      In &#91;1]: type('hello')\n      Out&#91;1]: &lt;type 'str'>\n    \n    indicates that the type name for strings is 'str'.\n    \n    ``%who`` always excludes executed names loaded through your configuration\n    file and things which are internal to IPython.\n    \n    This is deliberate, as typically you may load many modules and the\n    purpose of %who is to show you only what you've manually defined.\n    \n    Examples\n    --------\n    \n    Define two variables and list them with who::\n    \n      In &#91;1]: alpha = 123\n    \n      In &#91;2]: beta = 'test'\n    \n      In &#91;3]: %who\n      alpha   beta\n    \n      In &#91;4]: %who int\n      alpha\n    \n      In &#91;5]: %who str\n      beta\n%who_ls:\n    Return a sorted list of all interactive variables.\n    \n    If arguments are given, only variables of types matching these\n    arguments are returned.\n    \n    Examples\n    --------\n    Define two variables and list them with who_ls::\n    \n      In &#91;1]: alpha = 123\n    \n      In &#91;2]: beta = 'test'\n    \n      In &#91;3]: %who_ls\n      Out&#91;3]: &#91;'alpha', 'beta']\n    \n      In &#91;4]: %who_ls int\n      Out&#91;4]: &#91;'alpha']\n    \n      In &#91;5]: %who_ls str\n      Out&#91;5]: &#91;'beta']\n%whos:\n    Like %who, but gives some extra information about each variable.\n    \n    The same type filtering of %who can be applied here.\n    \n    For all variables, the type is printed. Additionally it prints:\n    \n      - For {},&#91;],(): their length.\n    \n      - For numpy arrays, a summary with shape, number of\n        elements, typecode and size in memory.\n    \n      - Everything else: a string representation, snipping their middle if\n        too long.\n    \n    Examples\n    --------\n    Define two variables and list them with whos::\n    \n      In &#91;1]: alpha = 123\n    \n      In &#91;2]: beta = 'test'\n    \n      In &#91;3]: %whos\n      Variable   Type        Data\/Info\n      --------------------------------\n      alpha      int         123\n      beta       str         test\n%xdel:\n    Delete a variable, trying to clear it from anywhere that\n    IPython's machinery has references to it. By default, this uses\n    the identity of the named object in the user namespace to remove\n    references held under other names. The object is also removed\n    from the output history.\n    \n    Options\n      -n : Delete the specified name from all namespaces, without\n      checking their identity.\n%xmode:\n    Switch modes for the exception handlers.\n    \n    Valid modes: Plain, Context, Verbose, and Minimal.\n    \n    If called without arguments, acts as a toggle.\n    \n    When in verbose mode the value `--show` (and `--hide`)\n    will respectively show (or hide) frames with ``__tracebackhide__ =\n    True`` value set.\n%%!:\n    Shell execute - run shell command and capture output (!! is short-hand).\n    \n    %sx command\n    \n    IPython will run the given command using commands.getoutput(), and\n    return the result formatted as a list (split on '\\n').  Since the\n    output is _returned_, it will be stored in ipython's regular output\n    cache Out&#91;N] and in the '_N' automatic variables.\n    \n    Notes:\n    \n    1) If an input line begins with '!!', then %sx is automatically\n    invoked.  That is, while::\n    \n      !ls\n    \n    causes ipython to simply issue system('ls'), typing::\n    \n      !!ls\n    \n    is a shorthand equivalent to::\n    \n      %sx ls\n    \n    2) %sx differs from %sc in that %sx automatically splits into a list,\n    like '%sc -l'.  The reason for this is to make it as easy as possible\n    to process line-oriented shell output via further python commands.\n    %sc is meant to provide much finer control, but requires more\n    typing.\n    \n    3) Just like %sc -l, this is a list with special attributes:\n    ::\n    \n      .l (or .list) : value as list.\n      .n (or .nlstr): value as newline-separated string.\n      .s (or .spstr): value as whitespace-separated string.\n    \n    This is very useful when trying to use such lists as arguments to\n    system commands.\n%%HTML:\n    Alias for `%%html`.\n%%SVG:\n    Alias for `%%svg`.\n%%bash:\n    %%bash script magic\n    \n    Run cells with bash in a subprocess.\n    \n    This is a shortcut for `%%script bash`\n%%capture:\n    ::\n    \n      %capture &#91;--no-stderr] &#91;--no-stdout] &#91;--no-display] &#91;output]\n    \n    run the cell, capturing stdout, stderr, and IPython's rich display() calls.\n    \n    positional arguments:\n      output        The name of the variable in which to store output. This is a\n                    utils.io.CapturedIO object with stdout\/err attributes for the\n                    text of the captured output. CapturedOutput also has a show()\n                    method for displaying the output, and __call__ as well, so you\n                    can use that to quickly display the output. If unspecified,\n                    captured output is discarded.\n    \n    options:\n      --no-stderr   Don't capture stderr.\n      --no-stdout   Don't capture stdout.\n      --no-display  Don't capture IPython's rich display.\n%%code_wrap:\n    ::\n    \n      %code_wrap &#91;--remove] &#91;--list] &#91;--list-all] &#91;name]\n    \n    Simple magic to quickly define a code transformer for all IPython's future input.\n    \n    ``__code__`` and ``__ret__`` are special variable that represent the code to run\n    and the value of the last expression of ``__code__`` respectively.\n    \n    Examples\n    --------\n    \n    .. ipython::\n    \n        In &#91;1]: %%code_wrap before_after\n           ...: print('before')\n           ...: __code__\n           ...: print('after')\n           ...: __ret__\n    \n        In &#91;2]: 1\n        before\n        after\n        Out&#91;2]: 1\n    \n        In &#91;3]: %code_wrap --list\n        before_after\n    \n        In &#91;4]: %code_wrap --list-all\n        before_after :\n            print('before')\n            __code__\n            print('after')\n            __ret__\n    \n        In &#91;5]: %code_wrap --remove before_after\n    \n    positional arguments:\n      name\n    \n    options:\n      --remove    remove the current transformer\n      --list      list existing transformers name\n      --list-all  list existing transformers name and code template\n%%debug:\n    ::\n    \n      %debug &#91;--breakpoint FILE:LINE] &#91;statement ...]\n    \n    Activate the interactive debugger.\n    \n    This magic command support two ways of activating debugger.\n    One is to activate debugger before executing code.  This way, you\n    can set a break point, to step through the code from the point.\n    You can use this mode by giving statements to execute and optionally\n    a breakpoint.\n    \n    The other one is to activate debugger in post-mortem mode.  You can\n    activate this mode simply running %debug without any argument.\n    If an exception has just occurred, this lets you inspect its stack\n    frames interactively.  Note that this will always work only on the last\n    traceback that occurred, so you must call this quickly after an\n    exception that you wish to inspect has fired, because if another one\n    occurs, it clobbers the previous one.\n    \n    If you want IPython to automatically do this on every exception, see\n    the %pdb magic for more details.\n    \n    .. versionchanged:: 7.3\n        When running code, user variables are no longer expanded,\n        the magic line is always left unmodified.\n    \n    positional arguments:\n      statement             Code to run in debugger. You can omit this in cell\n                            magic mode.\n    \n    options:\n      --breakpoint &lt;FILE:LINE>, -b &lt;FILE:LINE>\n                            Set break point at LINE in FILE.\n%%file:\n    Alias for `%%writefile`.\n%%html:\n    ::\n    \n      %html &#91;--isolated]\n    \n    Render the cell as a block of HTML\n    \n    options:\n      --isolated  Annotate the cell as 'isolated'. Isolated cells are rendered\n                  inside their own &lt;iframe> tag\n%%javascript:\n    Run the cell block of Javascript code\n    \n    Starting with IPython 8.0 %%javascript is pending deprecation to be replaced\n    by a more flexible system\n    \n    Please See https:\/\/github.com\/ipython\/ipython\/issues\/13376\n%%js:\n    Run the cell block of Javascript code\n    \n    Alias of `%%javascript`\n    \n    Starting with IPython 8.0 %%javascript is pending deprecation to be replaced\n    by a more flexible system\n    \n    Please See https:\/\/github.com\/ipython\/ipython\/issues\/13376\n%%latex:\n    Render the cell as a block of LaTeX\n    \n    The subset of LaTeX which is supported depends on the implementation in\n    the client.  In the Jupyter Notebook, this magic only renders the subset\n    of LaTeX defined by MathJax\n    &#91;here](https:\/\/docs.mathjax.org\/en\/v2.5-latest\/tex.html).\n%%markdown:\n    Render the cell as Markdown text block\n%%perl:\n    %%perl script magic\n    \n    Run cells with perl in a subprocess.\n    \n    This is a shortcut for `%%script perl`\n%%prun:\n    Run a statement through the python code profiler.\n    \n    Usage, in line mode:\n      %prun &#91;options] statement\n    \n    Usage, in cell mode:\n      %%prun &#91;options] &#91;statement]\n      code...\n      code...\n    \n    In cell mode, the additional code lines are appended to the (possibly\n    empty) statement in the first line.  Cell mode allows you to easily\n    profile multiline blocks without having to put them in a separate\n    function.\n    \n    The given statement (which doesn't require quote marks) is run via the\n    python profiler in a manner similar to the profile.run() function.\n    Namespaces are internally managed to work correctly; profile.run\n    cannot be used in IPython because it makes certain assumptions about\n    namespaces which do not hold under IPython.\n    \n    Options:\n    \n    -l &lt;limit>\n      you can place restrictions on what or how much of the\n      profile gets printed. The limit value can be:\n    \n         * A string: only information for function names containing this string\n           is printed.\n    \n         * An integer: only these many lines are printed.\n    \n         * A float (between 0 and 1): this fraction of the report is printed\n           (for example, use a limit of 0.4 to see the topmost 40% only).\n    \n      You can combine several limits with repeated use of the option. For\n      example, ``-l __init__ -l 5`` will print only the topmost 5 lines of\n      information about class constructors.\n    \n    -r\n      return the pstats.Stats object generated by the profiling. This\n      object has all the information about the profile in it, and you can\n      later use it for further analysis or in other functions.\n    \n    -s &lt;key>\n      sort profile by given key. You can provide more than one key\n      by using the option several times: '-s key1 -s key2 -s key3...'. The\n      default sorting key is 'time'.\n    \n      The following is copied verbatim from the profile documentation\n      referenced below:\n    \n      When more than one key is provided, additional keys are used as\n      secondary criteria when the there is equality in all keys selected\n      before them.\n    \n      Abbreviations can be used for any key names, as long as the\n      abbreviation is unambiguous.  The following are the keys currently\n      defined:\n    \n      ============  =====================\n      Valid Arg     Meaning\n      ============  =====================\n      \"calls\"       call count\n      \"cumulative\"  cumulative time\n      \"file\"        file name\n      \"module\"      file name\n      \"pcalls\"      primitive call count\n      \"line\"        line number\n      \"name\"        function name\n      \"nfl\"         name\/file\/line\n      \"stdname\"     standard name\n      \"time\"        internal time\n      ============  =====================\n    \n      Note that all sorts on statistics are in descending order (placing\n      most time consuming items first), where as name, file, and line number\n      searches are in ascending order (i.e., alphabetical). The subtle\n      distinction between \"nfl\" and \"stdname\" is that the standard name is a\n      sort of the name as printed, which means that the embedded line\n      numbers get compared in an odd way.  For example, lines 3, 20, and 40\n      would (if the file names were the same) appear in the string order\n      \"20\" \"3\" and \"40\".  In contrast, \"nfl\" does a numeric compare of the\n      line numbers.  In fact, sort_stats(\"nfl\") is the same as\n      sort_stats(\"name\", \"file\", \"line\").\n    \n    -T &lt;filename>\n      save profile results as shown on screen to a text\n      file. The profile is still shown on screen.\n    \n    -D &lt;filename>\n      save (via dump_stats) profile statistics to given\n      filename. This data is in a format understood by the pstats module, and\n      is generated by a call to the dump_stats() method of profile\n      objects. The profile is still shown on screen.\n    \n    -q\n      suppress output to the pager.  Best used with -T and\/or -D above.\n    \n    If you want to run complete programs under the profiler's control, use\n    ``%run -p &#91;prof_opts] filename.py &#91;args to program]`` where prof_opts\n    contains profiler specific options as described here.\n    \n    You can read the complete documentation for the profile module with::\n    \n      In &#91;1]: import profile; profile.help()\n    \n    .. versionchanged:: 7.3\n        User variables are no longer expanded,\n        the magic line is always left unmodified.\n%%pypy:\n    %%pypy script magic\n    \n    Run cells with pypy in a subprocess.\n    \n    This is a shortcut for `%%script pypy`\n%%python:\n    %%python script magic\n    \n    Run cells with python in a subprocess.\n    \n    This is a shortcut for `%%script python`\n%%python2:\n    %%python2 script magic\n    \n    Run cells with python2 in a subprocess.\n    \n    This is a shortcut for `%%script python2`\n%%python3:\n    %%python3 script magic\n    \n    Run cells with python3 in a subprocess.\n    \n    This is a shortcut for `%%script python3`\n%%ruby:\n    %%ruby script magic\n    \n    Run cells with ruby in a subprocess.\n    \n    This is a shortcut for `%%script ruby`\n%%script:\n    ::\n    \n      %shebang &#91;--no-raise-error] &#91;--proc PROC] &#91;--bg] &#91;--err ERR] &#91;--out OUT]\n    \n    Run a cell via a shell command\n    \n    The `%%script` line is like the #! line of script,\n    specifying a program (bash, perl, ruby, etc.) with which to run.\n    \n    The rest of the cell is run by that program.\n    \n    Examples\n    --------\n    ::\n    \n        In &#91;1]: %%script bash\n           ...: for i in 1 2 3; do\n           ...:   echo $i\n           ...: done\n        1\n        2\n        3\n    \n    options:\n      --no-raise-error  Whether you should raise an error message in addition to a\n                        stream on stderr if you get a nonzero exit code.\n      --proc PROC       The variable in which to store Popen instance. This is\n                        used only when --bg option is given.\n      --bg              Whether to run the script in the background. If given, the\n                        only way to see the output of the command is with\n                        --out\/err.\n      --err ERR         The variable in which to store stderr from the script. If\n                        the script is backgrounded, this will be the stderr\n                        *pipe*, instead of the stderr text itself and will not be\n                        autoclosed.\n      --out OUT         The variable in which to store stdout from the script. If\n                        the script is backgrounded, this will be the stdout\n                        *pipe*, instead of the stderr text itself and will not be\n                        auto closed.\n%%sh:\n    %%sh script magic\n    \n    Run cells with sh in a subprocess.\n    \n    This is a shortcut for `%%script sh`\n%%sql:\n    No documentation\n%%svg:\n    Render the cell as an SVG literal\n%%sx:\n    Shell execute - run shell command and capture output (!! is short-hand).\n    \n    %sx command\n    \n    IPython will run the given command using commands.getoutput(), and\n    return the result formatted as a list (split on '\\n').  Since the\n    output is _returned_, it will be stored in ipython's regular output\n    cache Out&#91;N] and in the '_N' automatic variables.\n    \n    Notes:\n    \n    1) If an input line begins with '!!', then %sx is automatically\n    invoked.  That is, while::\n    \n      !ls\n    \n    causes ipython to simply issue system('ls'), typing::\n    \n      !!ls\n    \n    is a shorthand equivalent to::\n    \n      %sx ls\n    \n    2) %sx differs from %sc in that %sx automatically splits into a list,\n    like '%sc -l'.  The reason for this is to make it as easy as possible\n    to process line-oriented shell output via further python commands.\n    %sc is meant to provide much finer control, but requires more\n    typing.\n    \n    3) Just like %sc -l, this is a list with special attributes:\n    ::\n    \n      .l (or .list) : value as list.\n      .n (or .nlstr): value as newline-separated string.\n      .s (or .spstr): value as whitespace-separated string.\n    \n    This is very useful when trying to use such lists as arguments to\n    system commands.\n%%system:\n    Shell execute - run shell command and capture output (!! is short-hand).\n    \n    %sx command\n    \n    IPython will run the given command using commands.getoutput(), and\n    return the result formatted as a list (split on '\\n').  Since the\n    output is _returned_, it will be stored in ipython's regular output\n    cache Out&#91;N] and in the '_N' automatic variables.\n    \n    Notes:\n    \n    1) If an input line begins with '!!', then %sx is automatically\n    invoked.  That is, while::\n    \n      !ls\n    \n    causes ipython to simply issue system('ls'), typing::\n    \n      !!ls\n    \n    is a shorthand equivalent to::\n    \n      %sx ls\n    \n    2) %sx differs from %sc in that %sx automatically splits into a list,\n    like '%sc -l'.  The reason for this is to make it as easy as possible\n    to process line-oriented shell output via further python commands.\n    %sc is meant to provide much finer control, but requires more\n    typing.\n    \n    3) Just like %sc -l, this is a list with special attributes:\n    ::\n    \n      .l (or .list) : value as list.\n      .n (or .nlstr): value as newline-separated string.\n      .s (or .spstr): value as whitespace-separated string.\n    \n    This is very useful when trying to use such lists as arguments to\n    system commands.\n%%time:\n    Time execution of a Python statement or expression.\n    \n    The CPU and wall clock times are printed, and the value of the\n    expression (if any) is returned.  Note that under Win32, system time\n    is always reported as 0, since it can not be measured.\n    \n    This function can be used both as a line and cell magic:\n    \n    - In line mode you can time a single-line statement (though multiple\n      ones can be chained with using semicolons).\n    \n    - In cell mode, you can time the cell body (a directly\n      following statement raises an error).\n    \n    This function provides very basic timing functionality.  Use the timeit\n    magic for more control over the measurement.\n    \n    .. versionchanged:: 7.3\n        User variables are no longer expanded,\n        the magic line is always left unmodified.\n    \n    Examples\n    --------\n    ::\n    \n      In &#91;1]: %time 2**128\n      CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s\n      Wall time: 0.00\n      Out&#91;1]: 340282366920938463463374607431768211456L\n    \n      In &#91;2]: n = 1000000\n    \n      In &#91;3]: %time sum(range(n))\n      CPU times: user 1.20 s, sys: 0.05 s, total: 1.25 s\n      Wall time: 1.37\n      Out&#91;3]: 499999500000L\n    \n      In &#91;4]: %time print('hello world')\n      hello world\n      CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s\n      Wall time: 0.00\n    \n    .. note::\n        The time needed by Python to compile the given expression will be\n        reported if it is more than 0.1s.\n    \n        In the example below, the actual exponentiation is done by Python\n        at compilation time, so while the expression can take a noticeable\n        amount of time to compute, that time is purely due to the\n        compilation::\n    \n            In &#91;5]: %time 3**9999;\n            CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s\n            Wall time: 0.00 s\n    \n            In &#91;6]: %time 3**999999;\n            CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s\n            Wall time: 0.00 s\n            Compiler : 0.78 s\n%%timeit:\n    Time execution of a Python statement or expression\n    \n    Usage, in line mode:\n      %timeit &#91;-n&lt;N> -r&lt;R> &#91;-t|-c] -q -p&lt;P> -o] statement\n    or in cell mode:\n      %%timeit &#91;-n&lt;N> -r&lt;R> &#91;-t|-c] -q -p&lt;P> -o] setup_code\n      code\n      code...\n    \n    Time execution of a Python statement or expression using the timeit\n    module.  This function can be used both as a line and cell magic:\n    \n    - In line mode you can time a single-line statement (though multiple\n      ones can be chained with using semicolons).\n    \n    - In cell mode, the statement in the first line is used as setup code\n      (executed but not timed) and the body of the cell is timed.  The cell\n      body has access to any variables created in the setup code.\n    \n    Options:\n    -n&lt;N>: execute the given statement &lt;N> times in a loop. If &lt;N> is not\n    provided, &lt;N> is determined so as to get sufficient accuracy.\n    \n    -r&lt;R>: number of repeats &lt;R>, each consisting of &lt;N> loops, and take the\n    average result.\n    Default: 7\n    \n    -t: use time.time to measure the time, which is the default on Unix.\n    This function measures wall time.\n    \n    -c: use time.clock to measure the time, which is the default on\n    Windows and measures wall time. On Unix, resource.getrusage is used\n    instead and returns the CPU user time.\n    \n    -p&lt;P>: use a precision of &lt;P> digits to display the timing result.\n    Default: 3\n    \n    -q: Quiet, do not print result.\n    \n    -o: return a TimeitResult that can be stored in a variable to inspect\n        the result in more details.\n    \n    .. versionchanged:: 7.3\n        User variables are no longer expanded,\n        the magic line is always left unmodified.\n    \n    Examples\n    --------\n    ::\n    \n      In &#91;1]: %timeit pass\n      8.26 ns \u00b1 0.12 ns per loop (mean \u00b1 std. dev. of 7 runs, 100000000 loops each)\n    \n      In &#91;2]: u = None\n    \n      In &#91;3]: %timeit u is None\n      29.9 ns \u00b1 0.643 ns per loop (mean \u00b1 std. dev. of 7 runs, 10000000 loops each)\n    \n      In &#91;4]: %timeit -r 4 u == None\n    \n      In &#91;5]: import time\n    \n      In &#91;6]: %timeit -n1 time.sleep(2)\n    \n    The times reported by %timeit will be slightly higher than those\n    reported by the timeit.py script when variables are accessed. This is\n    due to the fact that %timeit executes the statement in the namespace\n    of the shell, compared with timeit.py, which uses a single setup\n    statement to import function or create variables. Generally, the bias\n    does not matter as long as results from timeit.py are not mixed with\n    those from %timeit.\n%%writefile:\n    ::\n    \n      %writefile &#91;-a] filename\n    \n    Write the contents of the cell to a file.\n    \n    The file will be overwritten unless the -a (--append) flag is specified.\n    \n    positional arguments:\n      filename      file to write\n    \n    options:\n      -a, --append  Append contents of the cell to an existing file. The file will\n                    be created if it does not exist.\n\nSummary of magic functions (from %lsmagic):\nAvailable line magics:\n%alias  %alias_magic  %autoawait  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %code_wrap  %colors  %conda  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %mamba  %man  %matplotlib  %micromamba  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %pip  %popd  %pprint  %precision  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %restart_python  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode\n\nAvailable cell magics:\n%%!  %%HTML  %%SVG  %%bash  %%capture  %%code_wrap  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%sql  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile\n\nAutomagic is ON, % prefix IS NOT needed for line magics.<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Here\u2019s my Top 15 commands to try first \u2014 grouped into environment checks, Spark basics, and data handling so you learn in a logical order. 1\u20135: Environment&#8230; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-408","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/408","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=408"}],"version-history":[{"count":2,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/408\/revisions"}],"predecessor-version":[{"id":414,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/408\/revisions\/414"}],"wp:attachment":[{"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=408"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=408"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=408"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}