A practical, comprehensive reference for Python 3 covering fundamentals through advanced topics. Suitable for beginners, intermediate developers, advanced prog…
A practical, comprehensive reference for Python 3 covering fundamentals through advanced topics. Suitable for beginners, intermediate developers, advanced programmers, students, system administrators, data analysts, automation developers, and experienced Python developers needing a quick syntax and command reference. Targets actively supported Python 3 versions with modern conventions.
Table of Contents
Python Overview
Python is a high-level, general-purpose programming language emphasizing readability, simplicity, and developer productivity. It supports multiple paradigms: procedural, object-oriented, and functional.
Major Characteristics
- Dynamic typing
- Automatic memory management
- Extensive standard library
- Cross-platform
- Strong community and ecosystem
- Clear, readable syntax (often described as executable pseudocode)
Interpreted vs Compiled Execution
Python source is compiled to bytecode, which is then interpreted by a virtual machine. This hybrid model provides portability while retaining some compilation benefits. The process is transparent to the user in normal use.
Python Implementations
| Implementation |
Description |
| CPython |
Reference implementation; written in C; most widely used |
| PyPy |
JIT-compiled implementation; often significantly faster for long-running code |
| MicroPython |
Lightweight implementation for microcontrollers |
| IronPython |
Runs on .NET |
| Jython |
Runs on the JVM |
| Stackless |
Variant with microthreads |
CPython is the default and recommended starting point for most developers.
Python 3 vs Python 2
Python 2 reached end-of-life on January 1, 2020. All modern development uses Python 3. Key differences include print as a function, Unicode strings by default, integer division behavior, and many standard-library changes. This cheat sheet covers only Python 3.
Source Files, Bytecode, and the Interpreter
- Source files use the
.py extension.
- Bytecode is stored in
__pycache__ directories as .pyc files.
- The interpreter executes bytecode.
Running Python
python script.py # Run a script
python3 script.py # Explicit Python 3 on some systems
python -m module_name # Run a module as a script
python -c "print(42)" # Execute a statement
Interactive Interpreter (REPL)
python
>>> print("Hello")
Hello
__name__ == "__main__"
if __name__ == "__main__":
main()
This idiom ensures code runs only when the file is executed directly, not when imported.
How Python Code Is Executed
- Source is parsed into an abstract syntax tree (AST).
- The AST is compiled to bytecode.
- The Python Virtual Machine (PVM) interprets the bytecode.
- Objects are managed by reference counting plus a cyclic garbage collector.
Installation and Environment
Checking the Python Version
python --version
python3 --version
python -V
Locating the Executable
# Unix-like
which python3
type python3
# Windows (Command Prompt)
where python
# PowerShell
Get-Command python
Installing Python
Windows
- Download the official installer from python.org.
- Check “Add python.exe to PATH”.
- Use the Python Launcher (
py).
py -3.12 --version
py -3 script.py
macOS
- Prefer the official installer or Homebrew:
brew install python
Linux
# Debian/Ubuntu
sudo apt update
sudo apt install python3 python3-venv python3-pip
# Fedora
sudo dnf install python3
PATH Configuration
Ensure the Python installation directory and its Scripts (Windows) or bin (Unix) directory are on PATH.
Environment Variables
| Variable |
Purpose |
PATH |
Locates the interpreter and scripts |
PYTHONPATH |
Additional module search paths |
PYTHONHOME |
Alternative prefix for the installation |
PYTHONSTARTUP |
Script run at interactive startup |
sys.path
import sys
print(sys.path)
sys.path is initialized from the script directory, PYTHONPATH, and installation defaults.
Python Syntax Fundamentals
Indentation
Python uses indentation (spaces preferred; 4 spaces by convention) to define blocks. Mixing tabs and spaces is an error.
if True:
print("indented")
Comments
# Single-line comment
"""
Multi-line
string that can serve as a comment
"""
Statements and Expressions
- Statement: performs an action (
x = 1, print(x)).
- Expression: produces a value (
1 + 2, len(s)).
Variables and Assignment
x = 10
name = "Alice"
x, y = 1, 2 # Multiple assignment
a = b = c = 0 # Chained assignment
Constants are conventional (uppercase names); Python has no true constants.
Naming Conventions (PEP 8)
| Style |
Use |
snake_case |
Variables, functions, modules |
PascalCase |
Classes |
UPPER_SNAKE_CASE |
Constants |
_single_leading |
Internal use |
__double_leading |
Name mangling |
Identifiers and Keywords
Identifiers start with a letter or underscore and contain letters, digits, or underscores. Keywords cannot be used as identifiers:
import keyword
print(keyword.kwlist)
Operators
Arithmetic
+ - * / // % **
Comparison
== != < > <= >=
Logical
and or not
Membership
in not in
Identity
is is not
Bitwise
& | ^ ~ << >>
Operator Precedence (Highest to Lowest)
| Precedence |
Operators |
| Highest |
() |
|
** |
|
+x -x ~x |
|
* / // % |
|
+ - |
|
<< >> |
|
& |
|
^ |
|
` |
|
Comparisons, membership, identity |
|
not |
|
and |
| Lowest |
or |
Truthiness
Falsy values: None, False, 0, 0.0, 0j, empty sequences/collections ("", [], (), {}, set()).
Everything else is truthy.
if []:
print("never")
if [1]:
print("always")
None, is, and ==
x = None
if x is None: # Preferred for None
...
if x == None: # Works but not preferred
...
Use is for identity; == for equality of value.
Built-in Data Types
| Type |
Mutable |
Example |
int |
No |
42 |
float |
No |
3.14 |
complex |
No |
1+2j |
bool |
No |
True, False |
str |
No |
"hello" |
bytes |
No |
b"hello" |
bytearray |
Yes |
bytearray(b"hello") |
memoryview |
No |
memoryview(b"abc") |
list |
Yes |
[1, 2, 3] |
tuple |
No |
(1, 2, 3) |
range |
No |
range(10) |
dict |
Yes |
{"a": 1} |
set |
Yes |
{1, 2, 3} |
frozenset |
No |
frozenset({1, 2}) |
NoneType |
No |
None |
Integers
Arbitrary precision.
x = 1_000_000 # Underscores for readability
bin(10), oct(10), hex(10)
int("ff", 16)
Floats
IEEE 754 double precision. Be aware of precision issues.
0.1 + 0.2 # 0.30000000000000004
Complex
c = 3 + 4j
c.real, c.imag, abs(c)
Booleans
Subclass of int. True == 1, False == 0.
NoneType
Singleton representing the absence of a value.
Strings
Creation
s1 = 'single'
s2 = "double"
s3 = """triple
quoted"""
s4 = r"raw\nstring" # Raw string
Indexing and Slicing
s = "Python"
s[0] # 'P'
s[-1] # 'n'
s[1:4] # 'yth'
s[::2] # 'Pto'
s[::-1] # 'nohtyP'
Concatenation and Repetition
"Hello" + " " + "World"
"ha" * 3
Formatting
f-strings (Python 3.6+; preferred)
name = "Alice"
f"Hello, {name}!"
f"{42:#x}" # Format specifications
str.format()
"Hello, {}!".format(name)
"{0} {1}".format("a", "b")
"{name}".format(name="Alice")
% formatting (legacy)
"Hello, %s!" % name
Common String Methods
| Method |
Description |
upper() / lower() |
Case conversion |
strip() / lstrip() / rstrip() |
Remove whitespace |
split(sep=None) |
Split into list |
join(iterable) |
Join iterable with separator |
replace(old, new) |
Replace occurrences |
find(sub) / index(sub) |
Search (index raises if missing) |
startswith() / endswith() |
Prefix/suffix checks |
isdigit() / isalpha() / isalnum() |
Character tests |
encode() / decode() |
Encoding conversion |
translate() |
Character mapping |
" hello ".strip()
",".join(["a", "b", "c"])
"hello world".replace("world", "Python")
Numbers and Mathematics
Arithmetic Operators
10 / 3 # 3.333... (true division)
10 // 3 # 3 (floor division)
10 % 3 # 1
2 ** 10 # 1024
divmod(10, 3) # (3, 1)
pow(2, 10, 100) # modular exponentiation
Built-in Functions
abs(-5)
round(3.14159, 2)
min(1, 2, 3)
max(1, 2, 3)
sum([1, 2, 3])
math Module
import math
math.sqrt(16)
math.ceil(3.2)
math.floor(3.8)
math.pi
math.factorial(5)
decimal and fractions
from decimal import Decimal, getcontext
getcontext().prec = 50
Decimal("0.1") + Decimal("0.2")
from fractions import Fraction
Fraction(1, 3) + Fraction(1, 6)
Random Numbers
import random
random.random()
random.randint(1, 10)
random.choice([1, 2, 3])
random.shuffle(lst)
For cryptographic use prefer secrets.
Lists, Tuples, Sets, and Dictionaries
Lists
Mutable ordered sequences.
lst = [1, 2, 3]
lst.append(4)
lst.extend([5, 6])
lst.insert(0, 0)
lst.remove(3)
x = lst.pop()
lst.clear()
lst.index(2)
lst.count(1)
lst.sort()
lst.reverse()
lst.copy() # Shallow copy
lst[:] # Also shallow copy
Tuples
Immutable ordered sequences.
t = (1, 2, 3)
t = 1, 2, 3 # Parentheses optional
a, b, c = t # Unpacking
Named tuples
from collections import namedtuple
Point = namedtuple("Point", ["x", "y"])
p = Point(1, 2)
Sets
Unordered collections of unique hashable elements.
s = {1, 2, 3}
s.add(4)
s.remove(1) # Raises if missing
s.discard(1) # Silent if missing
s.union(other)
s.intersection(other)
s.difference(other)
s.symmetric_difference(other)
s.issubset(other)
s.issuperset(other)
Dictionaries
Mutable mapping of keys to values (insertion-ordered since Python 3.7).
d = {"a": 1, "b": 2}
d["c"] = 3
d.get("d", 0)
d.keys()
d.values()
d.items()
d.update({"e": 5})
d.setdefault("f", 6)
del d["a"]
Dictionary comprehensions
{x: x**2 for x in range(5)}
Type Conversion
int("42")
float("3.14")
str(42)
bool(1)
list("abc")
tuple([1, 2])
set([1, 1, 2])
dict([("a", 1), ("b", 2)])
bytes("hello", "utf-8")
bytearray(b"hello")
Implicit conversion occurs in some arithmetic contexts; prefer explicit conversion for clarity.
Common errors: ValueError on invalid conversions, TypeError on incompatible types.
Control Flow
Conditional Statements
if condition:
...
elif other:
...
else:
...
Conditional expression
x = a if condition else b
Loops
for item in iterable:
...
else:
# Executes if loop completed without break
while condition:
...
else:
...
break # Exit loop
continue # Next iteration
pass # No-op placeholder
Structural Pattern Matching (Python 3.10+)
match value:
case 0:
print("zero")
case 1 | 2:
print("one or two")
case [x, y]:
print(f"list of {x}, {y}")
case {"name": name}:
print(name)
case _:
print("default")
Comprehensions and Iteration
[x**2 for x in range(10) if x % 2 == 0] # List
{x**2 for x in range(10)} # Set
{x: x**2 for x in range(5)} # Dict
(x**2 for x in range(10)) # Generator expression
for i, value in enumerate(lst, start=1):
...
for a, b in zip(list1, list2):
...
iter() and next() work with any iterator. Generators and generator expressions are lazy.
Functions
Definition and Calling
def greet(name: str, greeting: str = "Hello") -> str:
"""Return a greeting."""
return f"{greeting}, {name}!"
Parameter Types
def f(pos, /, pos_or_kw, *, kw_only, **kwargs):
...
/ separates positional-only parameters (Python 3.8+).* separates keyword-only parameters.*args collects extra positional arguments as a tuple.**kwargs collects extra keyword arguments as a dict.
Return Values
def multi():
return 1, 2, 3 # Returns a tuple
a, b, c = multi()
Scope and Closures
def outer():
x = 10
def inner():
nonlocal x
x += 1
return x
return inner
Recursion
def factorial(n):
return 1 if n <= 1 else n * factorial(n - 1)
Python has a recursion limit (sys.getrecursionlimit()).
Lambda, Functional Programming, and Iteration Tools
square = lambda x: x**2
Prefer comprehensions or generator expressions over map/filter for readability in most cases.
from functools import reduce
reduce(lambda a, b: a + b, [1, 2, 3, 4])
sorted(lst, key=lambda x: x.lower())
any(x > 0 for x in lst)
all(x > 0 for x in lst)
Useful modules: itertools, functools, operator.
Scope and Namespaces
LEGB Rule
- Local
- Enclosing
- Global
- Built-in
x = "global"
def outer():
x = "enclosing"
def inner():
x = "local"
print(x)
inner()
Common Pitfalls
Mutable default arguments
def f(lst=[]): # Dangerous
lst.append(1)
return lst
Prefer None and create inside the function.
Late-binding closures
funcs = [lambda: i for i in range(3)] # All return 2
funcs = [lambda i=i: i for i in range(3)] # Correct
Object-Oriented Programming
class Point:
def __init__(self, x: float, y: float):
self.x = x
self.y = y
def distance(self) -> float:
return (self.x**2 + self.y**2) ** 0.5
@classmethod
def origin(cls):
return cls(0, 0)
@staticmethod
def is_origin(x, y):
return x == 0 and y == 0
@property
def magnitude(self):
return self.distance()
Inheritance
class ColoredPoint(Point):
def __init__(self, x, y, color):
super().__init__(x, y)
self.color = color
Dataclasses (Python 3.7+)
from dataclasses import dataclass, field
@dataclass
class Point:
x: float
y: float
label: str = field(default="")
Abstract Base Classes
from abc import ABC, abstractmethod
class Shape(ABC):
@abstractmethod
def area(self):
...
Duck typing: if it walks like a duck and quacks like a duck, it is treated as a duck.
Dunder and Special Methods
Dunder (double-underscore) methods, also called special methods or magic methods, allow classes to integrate with Python’s built-in syntax and protocols. Implementing the appropriate dunder methods makes custom objects behave like built-in types for operators, iteration, context management, attribute access, and more.
Only implement the methods required for the protocols you need. Over-implementing can hide bugs and reduce readability.
Object Creation and Initialization
__new__(cls, ...)
Purpose: Controls instance creation. Called before __init__. Rarely needed except for immutable types or singletons.
Returns: A new instance (usually super().__new__(cls)).
class Singleton:
_instance = None
def __new__(cls):
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance
__init__(self, ...)
Purpose: Initializes a newly created instance. The most commonly overridden dunder method.
Returns: Always None.
class Point:
def __init__(self, x: float, y: float):
self.x = x
self.y = y
__del__(self)
Purpose: Called when an object is about to be destroyed (finalizer). Not a destructor in the C++ sense; timing is not guaranteed. Avoid relying on it for critical cleanup—prefer context managers.
String Representation
__repr__(self)
Purpose: Unambiguous, ideally evaluable representation used by repr(), the interactive interpreter, and debugging.
Convention: Return a string that looks like a valid constructor call.
def __repr__(self):
return f"Point({self.x!r}, {self.y!r})"
__str__(self)
Purpose: Human-readable representation used by str() and print(). Falls back to __repr__ if not defined.
def __str__(self):
return f"({self.x}, {self.y})"
__format__(self, format_spec)
Purpose: Custom formatting for f-strings and str.format().
def __format__(self, spec):
if spec == "polar":
return f"r={self.magnitude:.2f}"
return f"({self.x}, {self.y})"
__bytes__(self)
Purpose: Convert to bytes via bytes(obj).
Comparison and Hashing
__eq__(self, other)
Purpose: Implements ==. Should return NotImplemented for unsupported types so the reflected operation can be tried.
def __eq__(self, other):
if not isinstance(other, Point):
return NotImplemented
return self.x == other.x and self.y == other.y
__ne__(self, other)
Purpose: Implements !=. Default implementation negates __eq__; override only if needed.
Ordering methods (__lt__, __le__, __gt__, __ge__)
Purpose: Implement <, <=, >, >=. Defining one does not automatically provide the others; use functools.total_ordering to generate the rest from __eq__ and one ordering method.
from functools import total_ordering
@total_ordering
class Version:
def __eq__(self, other): ...
def __lt__(self, other): ...
__hash__(self)
Purpose: Makes an object hashable (usable as dict key or set element). Must be consistent with __eq__: equal objects must have equal hashes. Mutable objects should set __hash__ = None.
def __hash__(self):
return hash((self.x, self.y))
Numeric and Arithmetic Emulation
Unary operators
| Method |
Operator |
Notes |
__neg__ |
-x |
Negation |
__pos__ |
+x |
Unary plus |
__abs__ |
abs(x) |
Absolute value |
__invert__ |
~x |
Bitwise inversion |
__round__ |
round(x) |
Rounding |
__floor__ / __ceil__ / __trunc__ |
math functions |
|
Binary arithmetic
| Method |
Operator |
Reflected |
__add__ |
+ |
__radd__ |
__sub__ |
- |
__rsub__ |
__mul__ |
* |
__rmul__ |
__truediv__ |
/ |
__rtruediv__ |
__floordiv__ |
// |
__rfloordiv__ |
__mod__ |
% |
__rmod__ |
__pow__ |
** |
__rpow__ |
__matmul__ |
@ |
__rmatmul__ |
In-place variants (__iadd__, __isub__, etc.) implement augmented assignment (+=, etc.). If not defined, Python falls back to the normal operator and rebinds the name.
def __add__(self, other):
if isinstance(other, Point):
return Point(self.x + other.x, self.y + other.y)
return NotImplemented
Conversion methods
__int__, __float__, __complex__, __index__ (for slicing and bin/hex/oct), __bool__.
def __bool__(self):
return bool(self.x or self.y)
Container and Sequence Emulation
__len__(self)
Purpose: Implements len(obj). Must return a non-negative integer.
__getitem__(self, key)
Purpose: Implements obj[key] (indexing, slicing, and mapping access).
__setitem__(self, key, value)
Purpose: Implements obj[key] = value.
__delitem__(self, key)
Purpose: Implements del obj[key].
__contains__(self, item)
Purpose: Implements item in obj. Falls back to iteration if not defined.
__iter__(self)
Purpose: Returns an iterator; makes the object iterable.
__reversed__(self)
Purpose: Implements reversed(obj). Prefer this over relying on __getitem__ + __len__ when a more efficient reverse is possible.
class MyList:
def __init__(self, data):
self._data = list(data)
def __len__(self):
return len(self._data)
def __getitem__(self, index):
return self._data[index]
def __setitem__(self, index, value):
self._data[index] = value
def __delitem__(self, index):
del self._data[index]
def __iter__(self):
return iter(self._data)
def __contains__(self, item):
return item in self._data
Iterator Protocol
__next__(self)
Purpose: Returns the next item or raises StopIteration. Used by the iterator protocol together with __iter__ returning self.
class Counter:
def __init__(self, low, high):
self.current = low
self.high = high
def __iter__(self):
return self
def __next__(self):
if self.current > self.high:
raise StopIteration
self.current += 1
return self.current - 1
Context Manager Protocol
__enter__(self)
Purpose: Enter the runtime context. The return value is bound to the as target.
__exit__(self, exc_type, exc_val, exc_tb)
Purpose: Exit the context. Return True to suppress an exception, False or None to propagate it.
class ManagedFile:
def __init__(self, path, mode="r"):
self.path = path
self.mode = mode
self.file = None
def __enter__(self):
self.file = open(self.path, self.mode, encoding="utf-8")
return self.file
def __exit__(self, exc_type, exc_val, exc_tb):
if self.file:
self.file.close()
return False # do not suppress exceptions
Callable Objects
__call__(self, ...)
Purpose: Makes an instance callable like a function: obj(...).
class Adder:
def __init__(self, n):
self.n = n
def __call__(self, x):
return x + self.n
add5 = Adder(5)
print(add5(10)) # 15
Attribute Access Customization
__getattr__(self, name)
Purpose: Called only when normal attribute lookup fails. Useful for lazy attributes or proxies.
__getattribute__(self, name)
Purpose: Called for every attribute access. Must be careful to avoid infinite recursion (use object.__getattribute__ or super()).
__setattr__(self, name, value)
Purpose: Called on every attribute assignment. Use object.__setattr__ or super() to set attributes safely.
__delattr__(self, name)
Purpose: Called on del obj.name.
__dir__(self)
Purpose: Customizes the result of dir(obj).
class Lazy:
def __getattr__(self, name):
if name == "value":
self.value = expensive_computation()
return self.value
raise AttributeError(name)
Descriptor Protocol
__get__(self, instance, owner)
__set__(self, instance, value)
__delete__(self, instance)
Purpose: Implement descriptors (used by property, classmethod, staticmethod, and custom managed attributes).
Asynchronous Protocols (Python 3.5+)
__aiter__ / __anext__ — asynchronous iteration__aenter__ / __aexit__ — asynchronous context managers__await__ — awaitable objects
class AsyncCounter:
def __init__(self, high):
self.high = high
self.current = 0
def __aiter__(self):
return self
async def __anext__(self):
if self.current >= self.high:
raise StopAsyncIteration
self.current += 1
return self.current - 1
Other Useful Dunder Methods
| Method |
Purpose |
__slots__ |
Restrict attributes and save memory |
__class_getitem__ |
Support generic subscripting (PEP 560) |
__init_subclass__ |
Customize subclass creation |
__set_name__ |
Called on descriptors when assigned in class |
__mro_entries__ |
Customize method resolution for generics |
__prepare__ |
Customize class namespace (metaclass) |
__instancecheck__ / __subclasscheck__ |
Custom isinstance/issubclass |
Practical Guidelines
- Prefer
__repr__ that is informative and, when practical, evaluable.
- Keep
__eq__ and __hash__ consistent; never make a mutable object hashable.
- Return
NotImplemented (not NotImplementedError) from binary operators when the type is unsupported.
- Use
@functools.total_ordering to reduce boilerplate for rich comparisons.
- Prefer context managers and generators over relying on
__del__.
- For most everyday classes,
__init__, __repr__, and possibly __eq__/__hash__ are sufficient; dataclasses generate many of these automatically.
Complete Minimal Example
from typing import Any
class Vector:
__slots__ = ("x", "y")
def __init__(self, x: float, y: float) -> None:
self.x = x
self.y = y
def __repr__(self) -> str:
return f"Vector({self.x!r}, {self.y!r})"
def __str__(self) -> str:
return f"<{self.x}, {self.y}>"
def __eq__(self, other: Any) -> bool:
if not isinstance(other, Vector):
return NotImplemented
return self.x == other.x and self.y == other.y
def __hash__(self) -> int:
return hash((self.x, self.y))
def __add__(self, other: "Vector") -> "Vector":
if not isinstance(other, Vector):
return NotImplemented
return Vector(self.x + other.x, self.y + other.y)
def __abs__(self) -> float:
return (self.x**2 + self.y**2) ** 0.5
def __bool__(self) -> bool:
return bool(self.x or self.y)
def __len__(self) -> int: # conceptual “dimension”
return 2
def __getitem__(self, index: int) -> float:
if index == 0:
return self.x
if index == 1:
return self.y
raise IndexError(index)
Exceptions and Error Handling
try:
risky()
except ValueError as e:
print(e)
except (TypeError, KeyError):
...
else:
# No exception
finally:
# Always runs
raise ValueError("message")
raise RuntimeError("new") from original_exception
Custom Exceptions
class MyError(Exception):
pass
Common Exceptions
| Exception |
Typical Cause |
SyntaxError |
Invalid syntax |
IndentationError |
Incorrect indentation |
NameError |
Undefined name |
TypeError |
Wrong type |
ValueError |
Correct type, wrong value |
AttributeError |
Missing attribute |
KeyError |
Missing dictionary key |
IndexError |
Sequence index out of range |
ImportError / ModuleNotFoundError |
Import failure |
FileNotFoundError |
Missing file |
PermissionError |
Insufficient permissions |
ZeroDivisionError |
Division by zero |
UnboundLocalError |
Local variable referenced before assignment |
RecursionError |
Exceeded recursion limit |
UnicodeDecodeError / UnicodeEncodeError |
Encoding issues |
JSONDecodeError |
Invalid JSON |
Best practice: catch specific exceptions; avoid bare except:.
File and Directory Handling
Prefer pathlib for modern code.
from pathlib import Path
p = Path("data.txt")
text = p.read_text(encoding="utf-8")
p.write_text("hello", encoding="utf-8")
with open("file.txt", encoding="utf-8") as f:
content = f.read()
Path("dir").mkdir(parents=True, exist_ok=True)
list(Path(".").glob("*.py"))
Path("a").rename("b")
Path("file").unlink()
os and shutil remain useful for lower-level or cross-version needs.
Always specify encoding explicitly when working with text.
JSON, CSV, and Common Data Formats
JSON
import json
data = json.loads(text)
text = json.dumps(obj, indent=2)
CSV
import csv
with open("data.csv", newline="", encoding="utf-8") as f:
reader = csv.DictReader(f)
for row in reader:
...
Pickle
import pickle
# Never unpickle untrusted data — arbitrary code execution risk
YAML requires third-party libraries (PyYAML, etc.).
Modules and Packages
import math
from math import sqrt as square_root
from package.module import name
Packages are directories containing __init__.py (optional in Python 3.3+ for namespace packages).
Absolute imports are preferred. Relative imports use leading dots.
if __name__ == "__main__":
...
Circular imports are usually resolved by moving imports inside functions or restructuring.
Virtual Environments and Package Management
python -m venv .venv
# Activate
source .venv/bin/activate # Unix
.venv\Scripts\activate # Windows CMD
.venv\Scripts\Activate.ps1 # PowerShell
deactivate
python -m pip install package
python -m pip install -r requirements.txt
python -m pip freeze > requirements.txt
python -m pip list
Prefer python -m pip over bare pip to avoid version mismatches.
Modern packaging uses pyproject.toml. pipx installs CLI tools into isolated environments.
Type Hints and Static Typing
from typing import Optional, Union, Callable, TypeVar, Protocol
from collections.abc import Iterable, Sequence
def process(items: list[str]) -> dict[str, int]:
...
Optional[str] # str | None (Python 3.10+)
list[str] # Python 3.9+
Static checkers: mypy, Pyright, Pyre.
Type hints improve documentation and tooling; they are optional at runtime (except with runtime-checked libraries).
Decorators
import functools
def my_decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
@my_decorator
def f():
...
Decorators with arguments require an extra nesting level. Stacking applies bottom-up.
Generators and Iterators
def gen():
yield 1
yield 2
g = gen()
next(g)
def chain(*iters):
for it in iters:
yield from it
Generators are memory-efficient for large or infinite sequences.
Context Managers
with open("file") as f:
...
from contextlib import contextmanager
@contextmanager
def managed():
# setup
try:
yield resource
finally:
# teardown
Regular Expressions
import re
re.search(r"\d+", text)
re.findall(r"\w+", text)
re.sub(r"old", "new", text)
re.split(r"\s+", text)
Prefer raw strings for patterns. Compile frequently used patterns with re.compile.
Dates and Times
from datetime import datetime, date, timedelta, timezone
from zoneinfo import ZoneInfo # Python 3.9+
now = datetime.now(timezone.utc)
dt = datetime.fromisoformat("2024-01-01T12:00:00+00:00")
Prefer timezone-aware datetimes. zoneinfo is the modern standard-library solution.
Standard Library Reference
| Module |
Purpose |
os / pathlib |
Operating-system and path operations |
sys |
Interpreter and system details |
shutil |
High-level file operations |
subprocess |
Spawn processes |
argparse |
Command-line parsing |
logging |
Flexible logging |
json / csv |
Data formats |
sqlite3 |
Embedded database |
datetime / time |
Date and time |
math / statistics / random / secrets |
Numeric and random |
re |
Regular expressions |
collections |
Specialized containers |
itertools / functools / operator |
Functional tools |
dataclasses / enum / typing / abc |
Modern language support |
contextlib |
Context-manager utilities |
unittest / doctest |
Testing |
hashlib / hmac / secrets |
Cryptographic primitives |
urllib / http / socket |
Networking |
tempfile / zipfile / tarfile / gzip |
Archives and temp files |
Consult the official documentation for full APIs.
Command-Line Python
python -c "print(42)"
python -m http.server 8000
python script.py arg1 arg2
import sys
print(sys.argv)
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("name")
args = parser.parse_args()
Exit codes: sys.exit(0) for success, non-zero for failure.
Subprocesses and Operating-System Interaction
import subprocess
result = subprocess.run(
["ls", "-l"],
capture_output=True,
text=True,
check=True,
timeout=30,
)
Prefer argument lists over shell=True to avoid injection risks. os.system() is discouraged for most new code.
Networking and HTTP
Standard library:
from urllib.request import urlopen
with urlopen("https://example.com", timeout=10) as resp:
data = resp.read()
Third-party libraries such as requests or httpx are commonly used for convenience; they are not part of the standard library.
Always set timeouts and validate certificates in production.
Databases
import sqlite3
with sqlite3.connect("app.db") as conn:
conn.execute("CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)")
conn.execute("INSERT INTO users (name) VALUES (?)", ("Alice",))
rows = conn.execute("SELECT * FROM users").fetchall()
Always use parameterized queries to prevent SQL injection. The same pattern (connect, execute with parameters, commit/rollback) applies to other database drivers.
Testing
import unittest
class TestMath(unittest.TestCase):
def test_add(self):
self.assertEqual(1 + 1, 2)
if __name__ == "__main__":
unittest.main()
from unittest.mock import Mock, patch
Pytest is a popular third-party alternative with concise syntax and rich plugins. Aim for unit, integration, and regression tests with meaningful coverage.
Debugging
breakpoint() # Python 3.7+; drops into pdb
python -m pdb script.py
Use logging instead of permanent print statements. Examine stack traces carefully; the last frame is usually the most relevant.
Logging
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
)
logging.info("message")
logging.exception("with traceback")
Configure loggers, handlers, and formatters for production. Avoid logging sensitive data.
Concurrency and Parallelism
| Approach |
Best for |
GIL impact |
| Threading |
I/O-bound |
Limited by GIL |
| Multiprocessing |
CPU-bound |
Bypasses GIL |
| Asyncio |
Concurrent I/O |
Single-threaded |
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
with ThreadPoolExecutor() as pool:
results = pool.map(func, items)
Protect shared state with locks. Avoid deadlocks by acquiring locks in a consistent order.
Asyncio
import asyncio
async def main():
await asyncio.sleep(1)
return 42
asyncio.run(main())
async def fetch():
...
async def main():
tasks = [asyncio.create_task(fetch()) for _ in range(10)]
results = await asyncio.gather(*tasks)
Async is for concurrent I/O, not CPU-bound parallelism. Blocking calls inside async code defeat the purpose.
Data Classes, Enums, and Modern Python Features
from enum import Enum, auto
class Color(Enum):
RED = auto()
GREEN = auto()
Assignment expressions (Python 3.8+):
if (n := len(items)) > 10:
...
Modern union syntax (Python 3.10+): str | None.
Structural pattern matching (Python 3.10+).
Memory Management and Internals
- Objects are reference-counted; cyclic garbage collector handles cycles.
id(obj) returns identity.- Mutable vs immutable affects sharing and hashing.
sys.getsizeof(obj) reports approximate size.- Weak references (
weakref) do not keep objects alive.
Performance Optimization
- Measure first (
timeit, cProfile, tracemalloc).
- Choose appropriate data structures (sets for membership, generators for large sequences).
- Avoid premature optimization.
- Cache pure functions with
@functools.lru_cache.
- Prefer algorithmic improvements over micro-optimizations.
Security
- Never use
eval or exec on untrusted input.
- Never unpickle untrusted data.
- Prefer argument lists over
shell=True.
- Validate and sanitize all external input.
- Use parameterized SQL queries.
- Store secrets in environment variables or secret managers, never in source.
- Pin dependencies and audit them.
- Use TLS for network communication.
- Prefer
secrets for cryptographic randomness.
Packaging and Distribution
Modern standard: pyproject.toml.
[project]
name = "mypackage"
version = "0.1.0"
dependencies = ["requests>=2.28"]
Build with a build backend (setuptools, hatchling, flit, etc.). Publish wheels and source distributions to PyPI. Entry points enable console scripts.
Documentation and Code Quality
- Write clear docstrings (Google, NumPy, or Sphinx style).
- Follow PEP 8.
- Use type hints where they add value.
- Format with Black or Ruff; lint with Ruff/Flake8; sort imports with isort or Ruff.
- Static analysis with mypy or Pyright.
- Maintain a README, changelog, and semantic versioning.
Python Project Structure
Small script
script.py
Library / application
mypackage/
__init__.py
module.py
tests/
test_module.py
pyproject.toml
README.md
.gitignore
Practical Real-World Examples
Hello World
print("Hello, World!")
Simple Calculator
def calculate(a: float, op: str, b: float) -> float:
ops = {"+": a + b, "-": a - b, "*": a * b, "/": a / b}
return ops[op]
File Reader with pathlib
from pathlib import Path
text = Path("data.txt").read_text(encoding="utf-8")
CSV Processor
import csv
from pathlib import Path
with Path("data.csv").open(newline="", encoding="utf-8") as f:
for row in csv.DictReader(f):
print(row["name"])
SQLite Example
import sqlite3
with sqlite3.connect(":memory:") as conn:
conn.execute("CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)")
conn.execute("INSERT INTO users (name) VALUES (?)", ("Alice",))
rows = conn.execute("SELECT * FROM users").fetchall()
Async Example
import asyncio
async def worker(n):
await asyncio.sleep(1)
return n * 2
async def main():
results = await asyncio.gather(*(worker(i) for i in range(5)))
print(results)
asyncio.run(main())
Unit Test
import unittest
class TestExample(unittest.TestCase):
def test_truth(self):
self.assertTrue(True)
Common Python Errors and Troubleshooting
| Error |
Meaning / Typical Fix |
SyntaxError |
Check parentheses, colons, quotes |
IndentationError |
Consistent 4-space indentation |
NameError |
Define the name before use; check spelling |
TypeError |
Check argument types and supported operations |
ValueError |
Validate input ranges and formats |
AttributeError |
Verify the object has the attribute |
KeyError |
Use .get() or check membership |
IndexError |
Check sequence length before indexing |
ModuleNotFoundError |
Install package or fix PYTHONPATH |
FileNotFoundError |
Verify path existence and working directory |
UnboundLocalError |
Assign before use or declare global/nonlocal |
RecursionError |
Add base case or increase limit carefully |
Common Python Mistakes
- Using
is for value comparison instead of ==
- Mutable default arguments
- Modifying a list while iterating over it
- Assuming shallow copy is deep
- Late-binding closures in loops
- Catching
Exception too broadly
- Forgetting to close files (use
with)
- Ignoring text encodings
- Hard-coding absolute paths
- Using
eval/pickle on untrusted data
- Blocking the event loop in async code
- Circular imports caused by poor structure
Python Quick Reference
Selected Built-in Functions
abs, all, any, enumerate, filter, map, max, min, next, open, print, range, reversed, round, sorted, sum, zip, len, type, isinstance, issubclass, id, hash, dir, help, vars
Virtual Environment & Package Commands
python -m venv .venv
source .venv/bin/activate
python -m pip install package
python -m pip freeze > requirements.txt
Testing & Debugging
python -m unittest
python -m pytest
python -m pdb script.py
Python by Task
| Task |
Approach |
| Read user input |
input("prompt") |
| Print formatted output |
print(f"{var}") |
| Convert types |
int(), str(), list(), etc. |
| Work with strings |
f-strings, methods, re |
| Work with lists |
indexing, methods, comprehensions |
| Read/write files |
pathlib or open + with |
| Process CSV/JSON |
csv, json modules |
| Create virtual environment |
python -m venv |
| Install package |
python -m pip install |
| Run tests |
unittest or pytest |
| Make HTTP request |
urllib or third-party requests/httpx |
| Concurrent I/O |
asyncio or ThreadPoolExecutor |
| CPU-bound parallelism |
ProcessPoolExecutor |
Beginner-to-Advanced Learning Path
- Syntax, variables, basic types, control flow
- Data structures and comprehensions
- Functions and scope
- Modules, packages, virtual environments
- File handling and exceptions
- Object-oriented programming
- Testing and debugging
- Type hints
- Generators, decorators, context managers
- Concurrency and asyncio
- Packaging, performance, security
- Professional practices (linting, CI, documentation)
Python Best Practices
- Follow PEP 8 and write readable code.
- Prefer clarity over cleverness (KISS, explicit is better than implicit).
- Keep functions small and focused.
- Handle errors specifically; log them.
- Write tests for non-trivial logic.
- Use virtual environments and pin dependencies.
- Document public APIs.
- Measure before optimizing.
- Treat security as a first-class concern.
- Keep learning; the language and ecosystem evolve.
Python Glossary
Argument — Value passed to a function when calling it.
Attribute — Value associated with an object, accessed via dot notation.
Class — Blueprint for creating objects.
Closure — Function that captures variables from an enclosing scope.
Coroutine — Function that can be paused and resumed (async def).
Decorator — Callable that modifies or wraps another callable.
Dictionary — Mutable mapping of keys to values.
Exception — Object representing an error or exceptional condition.
Generator — Iterator created by a function containing yield.
Hashable — Object with a stable hash value; required for dict keys and set elements.
Iterable — Object that can return an iterator (iter()).
Iterator — Object implementing __next__.
Lambda — Anonymous function expression.
Method — Function defined inside a class.
Module — File containing Python definitions and statements.
Namespace — Mapping from names to objects.
Object — Instance of a class; everything in Python is an object.
Package — Collection of modules organized in a directory hierarchy.
Parameter — Variable in a function definition that receives an argument.
Property — Managed attribute using the property decorator.
Scope — Region of a program where a name is visible.
Sequence — Ordered collection supporting indexing and slicing.
Serialization — Converting an object to a byte stream or text representation.
Thread — Lightweight unit of execution within a process.
Virtual environment — Isolated Python environment with its own packages.
Yield — Keyword that produces a value from a generator and suspends execution.