乐闻世界logo
搜索文章和话题

What are the features and use cases of Python metaprogramming?

2月21日 17:10

Python Metaprogramming Explained

Basic Concepts of Metaprogramming

Metaprogramming refers to writing code that can manipulate, generate, or modify code. Python provides rich metaprogramming tools, including decorators, metaclasses, dynamic attributes, etc.

Metaprogramming Use Cases

  • Framework development (e.g., Django ORM)
  • Code generation and automation
  • Dynamic property and method creation
  • Aspect-Oriented Programming (AOP)
  • Serialization and deserialization

Metaclasses

What is a Metaclass

A metaclass is a class that creates classes, just as a class is a template for creating objects, a metaclass is a template for creating classes.

python
# Basic concept
class MyClass:
    pass

# MyClass is an instance of type
print(type(MyClass))  # <class 'type'>

# obj is an instance of MyClass
obj = MyClass()
print(type(obj))  # <class '__main__.MyClass'>

Custom Metaclasses

python
class MyMeta(type):
    def __new__(cls, name, bases, namespace):
        # Execute when class is created
        print(f"Creating class: {name}")
        # Add class attribute
        namespace['created_by'] = 'MyMeta'
        return super().__new__(cls, name, bases, namespace)

class MyClass(metaclass=MyMeta):
    pass

print(MyClass.created_by)  # MyMeta

Metaclass Applications

python
class SingletonMeta(type):
    """Singleton metaclass"""
    _instances = {}
    
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]

class Singleton(metaclass=SingletonMeta):
    def __init__(self, value):
        self.value = value

s1 = Singleton(1)
s2 = Singleton(2)

print(s1 is s2)  # True
print(s1.value)  # 2

Advanced Metaclass Usage

python
class ValidateMeta(type):
    """Validation metaclass"""
    def __new__(cls, name, bases, namespace):
        # Ensure class has specific attributes
        if 'required_attr' not in namespace:
            raise TypeError(f"{name} must have 'required_attr'")
        
        # Validate methods
        for attr_name, attr_value in namespace.items():
            if callable(attr_value) and not attr_name.startswith('_'):
                if not hasattr(attr_value, '__annotations__'):
                    raise TypeError(f"Method {attr_name} must have type hints")
        
        return super().__new__(cls, name, bases, namespace)

class ValidatedClass(metaclass=ValidateMeta):
    required_attr = "value"
    
    def method(self, x: int) -> int:
        return x * 2

# class InvalidClass(metaclass=ValidateMeta):
#     pass  # TypeError: InvalidClass must have 'required_attr'

Dynamic Properties and Methods

Dynamic Properties

python
class DynamicAttributes:
    def __init__(self):
        self._data = {}
    
    def __getattr__(self, name):
        """Called when accessing non-existent attributes"""
        if name.startswith('get_'):
            attr_name = name[4:]
            return self._data.get(attr_name)
        raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")
    
    def __setattr__(self, name, value):
        """Called when setting attributes"""
        if name.startswith('_'):
            super().__setattr__(name, value)
        else:
            self._data[name] = value
    
    def __delattr__(self, name):
        """Called when deleting attributes"""
        if name in self._data:
            del self._data[name]
        else:
            super().__delattr__(name)

obj = DynamicAttributes()
obj.name = "Alice"
obj.age = 25

print(obj.get_name)  # Alice
print(obj.get_age)  # 25

Dynamic Methods

python
class DynamicMethods:
    def __init__(self):
        self.methods = {}
    
    def add_method(self, name, func):
        """Dynamically add methods"""
        self.methods[name] = func
    
    def __getattr__(self, name):
        """Dynamically call methods"""
        if name in self.methods:
            return self.methods[name]
        raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")

obj = DynamicMethods()

# Dynamically add methods
obj.add_method('greet', lambda self, name: f"Hello, {name}!")
obj.add_method('calculate', lambda self, x, y: x + y)

print(obj.greet("Alice"))  # Hello, Alice!
print(obj.calculate(3, 5))  # 8

Creating Methods with types Module

python
import types

class MyClass:
    pass

def new_method(self):
    return "This is a dynamically added method"

# Dynamically add method
MyClass.new_method = new_method

obj = MyClass()
print(obj.new_method())  # This is a dynamically added method

# Using types.MethodType
def another_method(self, value):
    return f"Value: {value}"

obj.another_method = types.MethodType(another_method, obj)
print(obj.another_method(42))  # Value: 42

Descriptors

Descriptor Protocol

Descriptors are classes that implement __get__, __set__, and __delete__ methods, used to control attribute access.

python
class Descriptor:
    def __init__(self, name=None):
        self.name = name
    
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return instance.__dict__.get(self.name, f"No {self.name} set")
    
    def __set__(self, instance, value):
        instance.__dict__[self.name] = value
    
    def __delete__(self, instance):
        if self.name in instance.__dict__:
            del instance.__dict__[self.name]

class Person:
    name = Descriptor('name')
    age = Descriptor('age')

person = Person()
person.name = "Alice"
person.age = 25

print(person.name)  # Alice
print(person.age)  # 25

Descriptor Applications

python
class ValidatedAttribute:
    """Validated attribute descriptor"""
    def __init__(self, validator=None, default=None):
        self.validator = validator
        self.default = default
        self.name = None
    
    def __set_name__(self, owner, name):
        self.name = f"_{name}"
    
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return getattr(instance, self.name, self.default)
    
    def __set__(self, instance, value):
        if self.validator and not self.validator(value):
            raise ValueError(f"Invalid value for {self.name}: {value}")
        setattr(instance, self.name, value)

class User:
    name = ValidatedAttribute(lambda x: isinstance(x, str) and len(x) > 0)
    age = ValidatedAttribute(lambda x: isinstance(x, int) and 0 <= x <= 150)
    email = ValidatedAttribute(lambda x: '@' in x)

user = User()
user.name = "Alice"
user.age = 25
user.email = "alice@example.com"

print(user.name)  # Alice
print(user.age)  # 25

# user.age = -5  # ValueError: Invalid value for _age: -5

Property Decorators

@property Decorator

python
class Temperature:
    def __init__(self, celsius):
        self._celsius = celsius
    
    @property
    def celsius(self):
        """Get Celsius temperature"""
        return self._celsius
    
    @celsius.setter
    def celsius(self, value):
        """Set Celsius temperature"""
        if value < -273.15:
            raise ValueError("Temperature below absolute zero")
        self._celsius = value
    
    @property
    def fahrenheit(self):
        """Get Fahrenheit temperature (read-only)"""
        return self._celsius * 9/5 + 32

temp = Temperature(25)
print(temp.celsius)  # 25
print(temp.fahrenheit)  # 77.0

temp.celsius = 30
print(temp.celsius)  # 30

# temp.fahrenheit = 100  # AttributeError: can't set attribute

Dynamic Property Calculation

python
class Circle:
    def __init__(self, radius):
        self._radius = radius
    
    @property
    def radius(self):
        return self._radius
    
    @radius.setter
    def radius(self, value):
        if value <= 0:
            raise ValueError("Radius must be positive")
        self._radius = value
    
    @property
    def diameter(self):
        return self._radius * 2
    
    @property
    def area(self):
        return 3.14159 * self._radius ** 2
    
    @property
    def circumference(self):
        return 2 * 3.14159 * self._radius

circle = Circle(5)
print(circle.diameter)  # 10
print(circle.area)  # 78.53975
print(circle.circumference)  # 31.4159

Dynamic Class Creation

Creating Classes with type

python
# Dynamically create class
def __init__(self, name):
    self.name = name

def greet(self):
    return f"Hello, {self.name}!"

# Create class using type
DynamicClass = type(
    'DynamicClass',
    (object,),
    {
        '__init__': __init__,
        'greet': greet,
        'class_var': 'dynamic'
    }
)

obj = DynamicClass("Alice")
print(obj.greet())  # Hello, Alice!
print(obj.class_var)  # dynamic

Dynamically Creating Subclasses

python
def create_subclass(base_class, subclass_name, extra_methods=None):
    """Dynamically create subclass"""
    namespace = extra_methods or {}
    return type(subclass_name, (base_class,), namespace)

class Base:
    def base_method(self):
        return "Base method"

# Dynamically create subclass
extra_methods = {
    'extra_method': lambda self: "Extra method"
}

SubClass = create_subclass(Base, 'SubClass', extra_methods)

obj = SubClass()
print(obj.base_method())  # Base method
print(obj.extra_method())  # Extra method

Class Decorators

Basic Class Decorator

python
def add_class_method(cls):
    """Decorator to add class methods"""
    @classmethod
    def class_method(cls):
        return f"Class method of {cls.__name__}"
    
    cls.class_method = class_method
    return cls

@add_class_method
class MyClass:
    pass

print(MyClass.class_method())  # Class method of MyClass

Class Decorator Applications

python
def singleton(cls):
    """Singleton class decorator"""
    instances = {}
    
    def get_instance(*args, **kwargs):
        if cls not in instances:
            instances[cls] = cls(*args, **kwargs)
        return instances[cls]
    
    return get_instance

@singleton
class Database:
    def __init__(self):
        self.connection = "Connected"

db1 = Database()
db2 = Database()

print(db1 is db2)  # True

Parameterized Class Decorators

python
def add_attributes(**attrs):
    """Decorator to add class attributes"""
    def decorator(cls):
        for name, value in attrs.items():
            setattr(cls, name, value)
        return cls
    return decorator

@add_attributes(version="1.0", author="Alice")
class MyClass:
    pass

print(MyClass.version)  # 1.0
print(MyClass.author)  # Alice

Practical Application Scenarios

1. ORM Framework

python
class Field:
    """Field descriptor"""
    def __init__(self, field_type, primary_key=False):
        self.field_type = field_type
        self.primary_key = primary_key
        self.name = None
    
    def __set_name__(self, owner, name):
        self.name = name
    
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return instance.__dict__.get(self.name)
    
    def __set__(self, instance, value):
        if not isinstance(value, self.field_type):
            raise TypeError(f"Expected {self.field_type}, got {type(value)}")
        instance.__dict__[self.name] = value

class ModelMeta(type):
    """Model metaclass"""
    def __new__(cls, name, bases, namespace):
        # Collect fields
        fields = {}
        for key, value in namespace.items():
            if isinstance(value, Field):
                fields[key] = value
        namespace['_fields'] = fields
        return super().__new__(cls, name, bases, namespace)

class Model(metaclass=ModelMeta):
    def __init__(self, **kwargs):
        for name, value in kwargs.items():
            setattr(self, name, value)

class User(Model):
    id = Field(int, primary_key=True)
    name = Field(str)
    age = Field(int)

user = User(id=1, name="Alice", age=25)
print(user.name)  # Alice
print(user.age)  # 25

2. API Response Validation

python
class ValidatedResponse:
    """Validated response class"""
    def __init__(self, schema):
        self.schema = schema
    
    def __call__(self, cls):
        def __init__(self, data):
            self.validate(data)
            for key, value in data.items():
                setattr(self, key, value)
        
        def validate(self, data):
            for field, field_type in self.schema.items():
                if field not in data:
                    raise ValueError(f"Missing field: {field}")
                if not isinstance(data[field], field_type):
                    raise TypeError(f"Invalid type for {field}")
        
        cls.__init__ = __init__
        cls.validate = validate
        return cls

@ValidatedResponse({'name': str, 'age': int, 'email': str})
class UserResponse:
    pass

user_data = {'name': 'Alice', 'age': 25, 'email': 'alice@example.com'}
user = UserResponse(user_data)
print(user.name)  # Alice

3. Dynamic Form Generation

python
class FormField:
    """Form field"""
    def __init__(self, field_type, required=False, default=None):
        self.field_type = field_type
        self.required = required
        self.default = default
        self.name = None
    
    def __set_name__(self, owner, name):
        self.name = name
    
    def validate(self, value):
        if self.required and value is None:
            raise ValueError(f"{self.name} is required")
        if value is not None and not isinstance(value, self.field_type):
            raise TypeError(f"Invalid type for {self.name}")
        return True

class FormMeta(type):
    """Form metaclass"""
    def __new__(cls, name, bases, namespace):
        fields = {}
        for key, value in namespace.items():
            if isinstance(value, FormField):
                fields[key] = value
        namespace['_fields'] = fields
        return super().__new__(cls, name, bases, namespace)

class Form(metaclass=FormMeta):
    def __init__(self, **kwargs):
        for name, field in self._fields.items():
            value = kwargs.get(name, field.default)
            field.validate(value)
            setattr(self, name, value)
    
    def to_dict(self):
        return {name: getattr(self, name) for name in self._fields}

class UserForm(Form):
    name = FormField(str, required=True)
    age = FormField(int, default=18)
    email = FormField(str, required=True)

form = UserForm(name="Alice", email="alice@example.com")
print(form.to_dict())  # {'name': 'Alice', 'age': 18, 'email': 'alice@example.com'}

Best Practices

1. Use Metaclasses Cautiously

python
# Bad practice - Overusing metaclasses
class ComplexMeta(type):
    def __new__(cls, name, bases, namespace):
        # Complex metaclass logic
        pass

# Good practice - Use class decorators
def add_functionality(cls):
    # Add functionality
    return cls

@add_functionality
class SimpleClass:
    pass

2. Prefer Descriptors Over getattr

python
# Good practice - Use descriptors
class ValidatedField:
    def __get__(self, instance, owner):
        return instance.__dict__.get(self.name)
    
    def __set__(self, instance, value):
        instance.__dict__[self.name] = value

class MyClass:
    field = ValidatedField()

# Bad practice - Use __getattr__
class BadClass:
    def __getattr__(self, name):
        return self.__dict__.get(name)

3. Provide Clear Documentation

python
class MyMeta(type):
    """Custom metaclass for adding class-level functionality
    
    This metaclass automatically adds a created_at attribute to all classes
    """
    def __new__(cls, name, bases, namespace):
        namespace['created_at'] = datetime.now()
        return super().__new__(cls, name, bases, namespace)

4. Consider Performance Impact

python
# Cache property access
class CachedProperty:
    def __init__(self, func):
        self.func = func
        self.name = func.__name__
    
    def __get__(self, instance, owner):
        if instance is None:
            return self
        if not hasattr(instance, f'_{self.name}'):
            setattr(instance, f'_{self.name}', self.func(instance))
        return getattr(instance, f'_{self.name}')

class MyClass:
    @CachedProperty
    def expensive_computation(self):
        # Expensive computation
        return sum(range(1000000))

Summary

Core concepts of Python metaprogramming:

  1. Metaclasses: Classes that create classes, controlling the class creation process
  2. Dynamic Properties: Using __getattr__, __setattr__ etc. to dynamically manage properties
  3. Dynamic Methods: Adding and modifying methods at runtime
  4. Descriptors: Controlling attribute access and modification
  5. Property Decorators: Using @property to create computed properties
  6. Dynamic Class Creation: Using type function to dynamically create classes
  7. Class Decorators: Modifying or enhancing class behavior

Metaprogramming use cases:

  • Framework development (ORM, form validation)
  • Code generation and automation
  • Dynamic API creation
  • Serialization and deserialization
  • Aspect-Oriented Programming

Metaprogramming considerations:

  • Use cautiously, avoid over-engineering
  • Provide clear documentation and examples
  • Consider performance impact
  • Prioritize simple solutions

Mastering metaprogramming techniques enables writing more flexible and powerful Python code.

标签：Python
热门标签
更多
Git(114)C语言(23)C++(15)React(34)前端(148)Cypress(27)JavaScript(56)Linux(20)Rust(10)Docker(65)Mongoose(18)TypeScript(27)MySQL(8)Vue(13)Tailwind CSS(29)CSS(19)ElasticSearch(24)网络(12)TypeORM(31)