Neuroevolution | API - Neurenix Documentation

← Previous Page

Overview¶

The Neuroevolution module provides implementations of evolutionary algorithms for optimizing neural network architectures and weights. Neuroevolution combines principles from evolutionary computation with neural networks, enabling the automatic discovery of effective network topologies and parameters without relying on gradient-based optimization.

This approach is particularly valuable for problems where gradient information is unavailable or unreliable, such as reinforcement learning tasks with sparse rewards, or when optimizing complex network architectures. The module implements various neuroevolution algorithms, including NEAT (NeuroEvolution of Augmenting Topologies), HyperNEAT, and evolutionary strategies like CMA-ES (Covariance Matrix Adaptation Evolution Strategy).

Key Concepts¶

Genetic Algorithms¶

Genetic algorithms are population-based optimization methods inspired by natural selection:

Individuals: Candidate solutions (neural networks)
Population: Collection of individuals
Fitness: Measure of solution quality
Selection: Process of choosing individuals for reproduction
Crossover: Combining genetic material from multiple parents
Mutation: Random changes to genetic material

NEAT (NeuroEvolution of Augmenting Topologies)¶

NEAT is an algorithm that evolves both the weights and structure of neural networks:

Direct Encoding: Networks are directly represented in the genome
Historical Markings: Track gene origins to enable meaningful crossover
Speciation: Protects innovation by grouping similar individuals
Complexification: Networks start simple and grow more complex over time

HyperNEAT¶

HyperNEAT extends NEAT to evolve large-scale networks with geometric regularities:

Indirect Encoding: Networks are generated by a separate network (CPPN)
Substrate: Geometric arrangement of neurons
Connectivity Patterns: Generated based on geometric relationships
Scalability: Can generate networks with millions of connections

Evolutionary Strategies¶

Evolutionary strategies focus on optimizing network weights:

CMA-ES: Adapts a multivariate normal distribution to efficiently search the parameter space
Natural Evolution Strategies: Uses gradient information estimated from the population
Population-Based Training: Combines evolutionary search with gradient-based optimization

Genetic Programming¶

Genetic programming evolves programs or expressions:

Tree-Based GP: Represents programs as trees
Cartesian GP: Represents programs as graphs
Grammar-Based GP: Uses grammars to constrain the search space

API Reference¶

Genetic Algorithm¶

neurenix.neuroevolution.GeneticAlgorithm(
    population_size: int = 100,
    mutation_rate: float = 0.1,
    crossover_rate: float = 0.7,
    selection_method: str = "tournament",
    elitism: int = 2
)

Creates a genetic algorithm for optimizing neural networks.

Parameters: - population_size: Number of individuals in the population - mutation_rate: Probability of mutation - crossover_rate: Probability of crossover - selection_method: Method for selecting individuals ("tournament", "roulette", "rank") - elitism: Number of best individuals to preserve unchanged

Methods: - initialize(genome_factory): Initialize the population - evolve(fitness_function, generations): Evolve the population - get_best_individual(): Get the best individual in the population - get_population(): Get the current population

Example:

from neurenix.neuroevolution import GeneticAlgorithm
from neurenix.nn import Sequential, Linear, ReLU

# Define a function to create neural networks
def create_network():
    return Sequential(
        Linear(10, 20),
        ReLU(),
        Linear(20, 5)
    )

# Define a fitness function
def evaluate_network(network):
    # Evaluate the network on some task
    # Return a fitness score
    return fitness_score

# Create a genetic algorithm
ga = GeneticAlgorithm(
    population_size=50,
    mutation_rate=0.1,
    crossover_rate=0.7,
    selection_method="tournament",
    elitism=2
)

# Initialize the population
ga.initialize(create_network)

# Evolve the population
best_network = ga.evolve(evaluate_network, generations=100)

# Get the best network
best_network = ga.get_best_individual()

NEAT¶

neurenix.neuroevolution.NEAT(
    input_size: int,
    output_size: int,
    population_size: int = 100,
    compatibility_threshold: float = 3.0,
    compatibility_disjoint_coefficient: float = 1.0,
    compatibility_weight_coefficient: float = 0.4,
    survival_threshold: float = 0.2
)

Creates a NEAT algorithm for evolving neural network topologies and weights.

Parameters: - input_size: Number of input neurons - output_size: Number of output neurons - population_size: Number of individuals in the population - compatibility_threshold: Threshold for speciation - compatibility_disjoint_coefficient: Coefficient for disjoint genes in compatibility calculation - compatibility_weight_coefficient: Coefficient for weight differences in compatibility calculation - survival_threshold: Fraction of each species allowed to reproduce

Methods: - initialize(): Initialize the population - evolve(fitness_function, generations): Evolve the population - get_best_individual(): Get the best individual in the population - get_species(): Get the current species

Example:

from neurenix.neuroevolution import NEAT

# Define a fitness function
def evaluate_network(network):
    # Evaluate the network on some task
    # Return a fitness score
    return fitness_score

# Create a NEAT algorithm
neat = NEAT(
    input_size=10,
    output_size=5,
    population_size=100,
    compatibility_threshold=3.0
)

# Initialize the population
neat.initialize()

# Evolve the population
neat.evolve(evaluate_network, generations=100)

# Get the best network
best_network = neat.get_best_individual()

HyperNEAT¶

neurenix.neuroevolution.HyperNEAT(
    substrate_dimensions: List[Tuple[int, int]],
    cppn_output_functions: List[str],
    population_size: int = 100,
    compatibility_threshold: float = 3.0
)

Creates a HyperNEAT algorithm for evolving large-scale neural networks with geometric regularities.

Parameters: - substrate_dimensions: Dimensions of each substrate layer - cppn_output_functions: Activation functions for CPPN outputs - population_size: Number of individuals in the population - compatibility_threshold: Threshold for speciation

Example:

from neurenix.neuroevolution import HyperNEAT

# Define a fitness function
def evaluate_network(network):
    # Evaluate the network on some task
    # Return a fitness score
    return fitness_score

# Create a HyperNEAT algorithm
hyperneat = HyperNEAT(
    substrate_dimensions=[(10, 1), (20, 1), (5, 1)],  # Input, hidden, output layers
    cppn_output_functions=["sigmoid", "tanh"],
    population_size=100
)

# Initialize the population
hyperneat.initialize()

# Evolve the population
hyperneat.evolve(evaluate_network, generations=100)

# Get the best CPPN
best_cppn = hyperneat.get_best_individual()

# Generate a substrate network from the CPPN
substrate_network = hyperneat.get_substrate_network(best_cppn)

CMA-ES¶

neurenix.neuroevolution.CMAES(
    model: neurenix.nn.Module,
    population_size: int = 16,
    sigma: float = 0.1,
    learning_rate: float = 0.1
)

Creates a CMA-ES algorithm for optimizing neural network weights.

Parameters: - model: Neural network model to optimize - population_size: Number of individuals in the population - sigma: Initial step size - learning_rate: Learning rate for updating the distribution

Methods: - initialize(): Initialize the algorithm - evolve(fitness_function, generations): Evolve the population - get_best_individual(): Get the best individual in the population - get_distribution_parameters(): Get the current distribution parameters

Example:

from neurenix.neuroevolution import CMAES
from neurenix.nn import Sequential, Linear, ReLU

# Create a neural network
model = Sequential(
    Linear(10, 20),
    ReLU(),
    Linear(20, 5)
)

# Define a fitness function
def evaluate_network(network):
    # Evaluate the network on some task
    # Return a fitness score
    return fitness_score

# Create a CMA-ES algorithm
cmaes = CMAES(
    model=model,
    population_size=16,
    sigma=0.1
)

# Initialize the algorithm
cmaes.initialize()

# Evolve the population
cmaes.evolve(evaluate_network, generations=100)

# Get the best network
best_network = cmaes.get_best_individual()

Population¶

neurenix.neuroevolution.Population(
    size: int,
    genome_factory: Callable,
    fitness_function: Callable
)

Creates a population of individuals for evolutionary algorithms.

Parameters: - size: Number of individuals in the population - genome_factory: Function to create new genomes - fitness_function: Function to evaluate genomes

Methods: - initialize(): Initialize the population - evaluate(): Evaluate all individuals in the population - select(method, count): Select individuals for reproduction - crossover(parent1, parent2): Perform crossover between two parents - mutate(individual, rate): Mutate an individual - get_best_individual(): Get the best individual in the population - get_average_fitness(): Get the average fitness of the population

Example:

from neurenix.neuroevolution import Population

# Define a function to create genomes
def create_genome():
    # Create a new genome
    return genome

# Define a fitness function
def evaluate_genome(genome):
    # Evaluate the genome
    return fitness_score

# Create a population
population = Population(
    size=100,
    genome_factory=create_genome,
    fitness_function=evaluate_genome
)

# Initialize the population
population.initialize()

# Evaluate the population
population.evaluate()

# Select individuals for reproduction
parents = population.select(method="tournament", count=10)

# Perform crossover
child = population.crossover(parents[0], parents[1])

# Mutate the child
mutated_child = population.mutate(child, rate=0.1)

# Get the best individual
best_individual = population.get_best_individual()

Genome¶

neurenix.neuroevolution.Genome(
    genes: List[Any] = None,
    fitness: float = 0.0
)

Represents a genome for evolutionary algorithms.

Parameters: - genes: List of genes - fitness: Fitness value

Methods: - crossover(other): Perform crossover with another genome - mutate(rate): Mutate the genome - distance(other): Calculate distance to another genome - copy(): Create a copy of the genome - to_network(): Convert the genome to a neural network

Example:

from neurenix.neuroevolution import Genome

# Create a genome
genome1 = Genome(genes=[1, 2, 3, 4, 5])
genome2 = Genome(genes=[5, 4, 3, 2, 1])

# Perform crossover
child = genome1.crossover(genome2)

# Mutate the child
child.mutate(rate=0.1)

# Calculate distance between genomes
distance = genome1.distance(genome2)

# Convert to a neural network
network = genome1.to_network()

Species¶

neurenix.neuroevolution.Species(
    representative: Genome,
    compatibility_threshold: float = 3.0
)

Represents a species in the NEAT algorithm.

Parameters: - representative: Representative genome for the species - compatibility_threshold: Threshold for determining membership

Methods: - add(genome): Add a genome to the species - remove(genome): Remove a genome from the species - contains(genome): Check if a genome belongs to the species - get_adjusted_fitness(): Get the adjusted fitness of the species - select_representative(): Select a new representative for the species - cull(percentage): Remove the worst individuals from the species

Example:

from neurenix.neuroevolution import Species, Genome

# Create a representative genome
representative = Genome(genes=[1, 2, 3, 4, 5])

# Create a species
species = Species(
    representative=representative,
    compatibility_threshold=3.0
)

# Create another genome
genome = Genome(genes=[1, 2, 4, 5, 6])

# Check if the genome belongs to the species
if species.contains(genome):
    # Add the genome to the species
    species.add(genome)

# Get the adjusted fitness of the species
adjusted_fitness = species.get_adjusted_fitness()

# Select a new representative
species.select_representative()

# Cull the worst individuals
species.cull(percentage=0.5)

Genetic Programming¶

neurenix.neuroevolution.GeneticProgramming(
    function_set: List[Callable],
    terminal_set: List[Any],
    population_size: int = 100,
    max_depth: int = 6,
    method: str = "tree"
)

Creates a genetic programming algorithm for evolving programs or expressions.

Parameters: - function_set: Set of functions to use in the programs - terminal_set: Set of terminals (variables, constants) to use in the programs - population_size: Number of individuals in the population - max_depth: Maximum depth of the program trees - method: Method for representing programs ("tree", "cartesian", "grammar")

Methods: - initialize(): Initialize the population - evolve(fitness_function, generations): Evolve the population - get_best_program(): Get the best program in the population - evaluate_program(program, inputs): Evaluate a program on inputs

Example:

import numpy as np
from neurenix.neuroevolution import GeneticProgramming

# Define function set
def add(x, y): return x + y
def sub(x, y): return x - y
def mul(x, y): return x * y
def div(x, y): return x / y if y != 0 else 1

function_set = [add, sub, mul, div]
terminal_set = ["x", "y", 1, 2, 3]

# Define fitness function
def evaluate_program(program, target_function):
    errors = []
    for x in np.linspace(-1, 1, 20):
        for y in np.linspace(-1, 1, 20):
            expected = target_function(x, y)
            actual = program({"x": x, "y": y})
            errors.append((expected - actual) ** 2)
    return -np.mean(errors)  # Negative because we maximize fitness

# Target function to approximate
def target_function(x, y):
    return x**2 + y**2

# Create a genetic programming algorithm
gp = GeneticProgramming(
    function_set=function_set,
    terminal_set=terminal_set,
    population_size=100,
    max_depth=6,
    method="tree"
)

# Initialize the population
gp.initialize()

# Evolve the population
gp.evolve(lambda p: evaluate_program(p, target_function), generations=100)

# Get the best program
best_program = gp.get_best_program()

Framework Comparison¶

Neurenix vs. TensorFlow¶

Feature	Neurenix	TensorFlow
Neuroevolution Support	Comprehensive (NEAT, HyperNEAT, CMA-ES)	Limited (basic genetic algorithms)
Topology Evolution	Native support	No native support
Integration with NN API	Seamless	Requires custom implementation
Speciation	Built-in	Not available
Distributed Evolution	Supported	Limited support
Visualization Tools	Comprehensive	Limited

Neurenix provides more comprehensive neuroevolution capabilities compared to TensorFlow, with built-in support for advanced algorithms like NEAT and HyperNEAT. TensorFlow requires custom implementations for most neuroevolution features, making it less accessible for users interested in evolutionary approaches to neural network optimization.

Neurenix vs. PyTorch¶

Feature	Neurenix	PyTorch
Neuroevolution Support	Comprehensive (NEAT, HyperNEAT, CMA-ES)	No native support
Topology Evolution	Native support	Requires third-party libraries
Integration with NN API	Seamless	Requires custom implementation
Speciation	Built-in	Not available natively
Distributed Evolution	Supported	Requires custom implementation
Visualization Tools	Comprehensive	Limited

PyTorch does not provide native support for neuroevolution, requiring users to rely on third-party libraries or custom implementations. Neurenix's integrated neuroevolution module offers a more cohesive experience, with seamless integration with the rest of the framework and built-in support for various evolutionary algorithms.

Neurenix vs. Scikit-Learn¶

Feature	Neurenix	Scikit-Learn
Neuroevolution Support	Comprehensive	No support
Neural Network Support	Native	Limited
Evolutionary Algorithms	Specialized for NN	Generic genetic algorithms
Topology Evolution	Supported	Not supported
Integration with Deep Learning	Seamless	Limited
Visualization Tools	Comprehensive	Limited

Scikit-Learn provides some basic genetic algorithm implementations but lacks support for neuroevolution and topology evolution. Neurenix fills this gap with its comprehensive neuroevolution module, enabling the evolution of both network weights and topologies for a wide range of applications.

Best Practices¶

Algorithm Selection¶

Choose the appropriate neuroevolution algorithm based on your task:

from neurenix.neuroevolution import NEAT, HyperNEAT, CMAES

# For tasks requiring topology evolution
neat = NEAT(
    input_size=10,
    output_size=5,
    population_size=100
)

# For tasks with geometric regularities
hyperneat = HyperNEAT(
    substrate_dimensions=[(10, 1), (20, 1), (5, 1)],
    cppn_output_functions=["sigmoid", "tanh"],
    population_size=100
)

# For tasks where only weight optimization is needed
cmaes = CMAES(
    model=model,
    population_size=16,
    sigma=0.1
)

Population Size¶

Choose an appropriate population size based on the complexity of your task:

from neurenix.neuroevolution import NEAT

# For simple tasks
simple_neat = NEAT(
    input_size=5,
    output_size=2,
    population_size=50  # Smaller population
)

# For complex tasks
complex_neat = NEAT(
    input_size=20,
    output_size=10,
    population_size=200  # Larger population
)

Fitness Function Design¶

Design effective fitness functions:

# Simple fitness function
def simple_fitness(network):
    # Evaluate on a single task
    return score

# Multi-objective fitness function
def multi_objective_fitness(network):
    # Evaluate on multiple objectives
    score1 = evaluate_objective1(network)
    score2 = evaluate_objective2(network)
    score3 = evaluate_objective3(network)

    # Combine scores (weighted sum, Pareto dominance, etc.)
    return 0.5 * score1 + 0.3 * score2 + 0.2 * score3

# Novelty-based fitness function
def novelty_fitness(network, archive):
    # Calculate behavioral distance to archive
    distances = [behavioral_distance(network, ind) for ind in archive]

    # Return average distance to k-nearest neighbors
    return sum(sorted(distances)[:15]) / 15

Hyperparameter Tuning¶

Tune hyperparameters for better performance:

from neurenix.neuroevolution import NEAT

# Start with default parameters
neat = NEAT(
    input_size=10,
    output_size=5,
    population_size=100,
    compatibility_threshold=3.0,
    compatibility_disjoint_coefficient=1.0,
    compatibility_weight_coefficient=0.4,
    survival_threshold=0.2
)

# Adjust parameters based on observations
# If species are too fragmented:
neat = NEAT(
    input_size=10,
    output_size=5,
    population_size=100,
    compatibility_threshold=4.0,  # Increase threshold
    compatibility_disjoint_coefficient=1.0,
    compatibility_weight_coefficient=0.4,
    survival_threshold=0.2
)

# If evolution is too slow:
neat = NEAT(
    input_size=10,
    output_size=5,
    population_size=100,
    compatibility_threshold=3.0,
    compatibility_disjoint_coefficient=1.0,
    compatibility_weight_coefficient=0.4,
    survival_threshold=0.3  # Increase survival rate
)

Tutorials¶

Evolving Neural Networks with NEAT¶

import neurenix as nx
from neurenix.neuroevolution import NEAT
import numpy as np

# Define a simple XOR problem
inputs = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
outputs = np.array([[0], [1], [1], [0]])

# Define a fitness function
def evaluate_network(network):
    total_error = 0
    for i in range(len(inputs)):
        input_tensor = nx.Tensor(inputs[i])
        output = network(input_tensor).item()
        target = outputs[i][0]
        total_error += (output - target) ** 2

    # Convert error to fitness (higher is better)
    fitness = 4.0 - total_error
    return max(0.0, fitness)  # Ensure non-negative fitness

# Create a NEAT algorithm
neat = NEAT(
    input_size=2,
    output_size=1,
    population_size=150,
    compatibility_threshold=3.0
)

# Initialize the population
neat.initialize()

# Evolve the population
best_fitness_history = []
for generation in range(100):
    # Evolve one generation
    neat.evolve(evaluate_network, generations=1)

    # Get the best individual
    best_individual = neat.get_best_individual()
    best_fitness = evaluate_network(best_individual)
    best_fitness_history.append(best_fitness)

    # Print progress
    print(f"Generation {generation}: Best Fitness = {best_fitness}")

    # Check if we've solved the problem
    if best_fitness > 3.9:  # Close enough to perfect
        print("Solution found!")
        break

# Test the best network
best_network = neat.get_best_individual()
print("\nTesting the best network:")
for i in range(len(inputs)):
    input_tensor = nx.Tensor(inputs[i])
    output = best_network(input_tensor).item()
    print(f"Input: {inputs[i]}, Output: {output:.4f}, Target: {outputs[i][0]}")

# Plot the fitness history
import matplotlib.pyplot as plt
plt.figure(figsize=(10, 6))
plt.plot(best_fitness_history)
plt.title("Best Fitness over Generations")
plt.xlabel("Generation")
plt.ylabel("Fitness")
plt.grid(True)
plt.savefig("neat_fitness_history.png")

Optimizing Neural Network Weights with CMA-ES¶

import neurenix as nx
from neurenix.nn import Sequential, Linear, ReLU
from neurenix.neuroevolution import CMAES
import numpy as np

# Create a dataset
np.random.seed(42)
X = np.random.rand(100, 10)
y = np.sin(X.sum(axis=1)) + 0.1 * np.random.randn(100)

# Convert to tensors
X_tensor = nx.Tensor(X)
y_tensor = nx.Tensor(y).reshape(-1, 1)

# Create a neural network
model = Sequential(
    Linear(10, 20),
    ReLU(),
    Linear(20, 20),
    ReLU(),
    Linear(20, 1)
)

# Define a fitness function
def evaluate_network(network):
    # Forward pass
    predictions = network(X_tensor)

    # Calculate MSE
    mse = ((predictions - y_tensor) ** 2).mean().item()

    # Convert to fitness (higher is better)
    return -mse  # Negative because we maximize fitness

# Create a CMA-ES algorithm
cmaes = CMAES(
    model=model,
    population_size=16,
    sigma=0.1
)

# Initialize the algorithm
cmaes.initialize()

# Evolve the population
best_fitness_history = []
for generation in range(100):
    # Evolve one generation
    cmaes.evolve(evaluate_network, generations=1)

    # Get the best individual
    best_individual = cmaes.get_best_individual()
    best_fitness = evaluate_network(best_individual)
    best_fitness_history.append(best_fitness)

    # Print progress
    print(f"Generation {generation}: Best Fitness = {best_fitness}")

    # Check if we've reached a good solution
    if best_fitness > -0.01:  # MSE < 0.01
        print("Good solution found!")
        break

# Test the best network
best_network = cmaes.get_best_individual()
predictions = best_network(X_tensor).numpy().flatten()

# Plot the results
import matplotlib.pyplot as plt
plt.figure(figsize=(12, 10))

# Plot predictions vs targets
plt.subplot(2, 1, 1)
plt.scatter(y, predictions)
plt.plot([min(y), max(y)], [min(y), max(y)], 'r--')
plt.title("Predictions vs Targets")
plt.xlabel("Targets")
plt.ylabel("Predictions")
plt.grid(True)

# Plot fitness history
plt.subplot(2, 1, 2)
plt.plot([-f for f in best_fitness_history])  # Convert back to MSE
plt.title("MSE over Generations")
plt.xlabel("Generation")
plt.ylabel("Mean Squared Error")
plt.yscale('log')
plt.grid(True)

plt.tight_layout()
plt.savefig("cmaes_results.png")

Evolving Large-Scale Networks with HyperNEAT¶

import neurenix as nx
from neurenix.neuroevolution import HyperNEAT
import numpy as np

# Define a simple task: learning a 2D function
def target_function(x, y):
    return np.sin(x * y) * np.cos(x + y)

# Generate a dataset
resolution = 16
x = np.linspace(-1, 1, resolution)
y = np.linspace(-1, 1, resolution)
X, Y = np.meshgrid(x, y)
Z = target_function(X, Y)

# Define a fitness function
def evaluate_network(network):
    total_error = 0
    for i in range(resolution):
        for j in range(resolution):
            # Create input coordinates
            input_tensor = nx.Tensor([x[i], y[j]])

            # Get network output
            output = network(input_tensor).item()

            # Calculate error
            target = Z[j, i]  # Note: meshgrid swaps indices
            total_error += (output - target) ** 2

    # Convert error to fitness (higher is better)
    mean_error = total_error / (resolution * resolution)
    fitness = 1.0 / (1.0 + mean_error)
    return fitness

# Create a HyperNEAT algorithm
hyperneat = HyperNEAT(
    substrate_dimensions=[(2, 1), (8, 8), (1, 1)],  # Input, hidden, output layers
    cppn_output_functions=["sigmoid", "tanh"],
    population_size=100
)

# Initialize the population
hyperneat.initialize()

# Evolve the population
best_fitness_history = []
for generation in range(200):
    # Evolve one generation
    hyperneat.evolve(evaluate_network, generations=1)

    # Get the best individual
    best_cppn = hyperneat.get_best_individual()
    best_network = hyperneat.get_substrate_network(best_cppn)
    best_fitness = evaluate_network(best_network)
    best_fitness_history.append(best_fitness)

    # Print progress
    print(f"Generation {generation}: Best Fitness = {best_fitness}")

    # Check if we've reached a good solution
    if best_fitness > 0.95:  # Close enough to perfect
        print("Good solution found!")
        break

# Generate predictions from the best network
best_cppn = hyperneat.get_best_individual()
best_network = hyperneat.get_substrate_network(best_cppn)

predictions = np.zeros((resolution, resolution))
for i in range(resolution):
    for j in range(resolution):
        input_tensor = nx.Tensor([x[i], y[j]])
        predictions[j, i] = best_network(input_tensor).item()

# Plot the results
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

plt.figure(figsize=(18, 6))

# Plot target function
ax1 = plt.subplot(1, 3, 1, projection='3d')
surf1 = ax1.plot_surface(X, Y, Z, cmap='viridis')
ax1.set_title("Target Function")
ax1.set_xlabel("X")
ax1.set_ylabel("Y")
ax1.set_zlabel("Z")

# Plot network output
ax2 = plt.subplot(1, 3, 2, projection='3d')
surf2 = ax2.plot_surface(X, Y, predictions, cmap='viridis')
ax2.set_title("Network Output")
ax2.set_xlabel("X")
ax2.set_ylabel("Y")
ax2.set_zlabel("Z")

# Plot fitness history
ax3 = plt.subplot(1, 3, 3)
ax3.plot(best_fitness_history)
ax3.set_title("Fitness over Generations")
ax3.set_xlabel("Generation")
ax3.set_ylabel("Fitness")
ax3.grid(True)

plt.tight_layout()
plt.savefig("hyperneat_results.png")

This documentation provides a comprehensive overview of the Neuroevolution module in Neurenix, including key concepts, API reference, framework comparisons, best practices, and tutorials for evolving neural networks using various evolutionary algorithms.

Page

Table Of Contents

Neuroevolution API Documentation¶

Overview¶

Key Concepts¶

Genetic Algorithms¶

NEAT (NeuroEvolution of Augmenting Topologies)¶

HyperNEAT¶

Evolutionary Strategies¶

Genetic Programming¶

API Reference¶

Genetic Algorithm¶

NEAT¶

HyperNEAT¶

CMA-ES¶

Population¶

Genome¶

Species¶

Genetic Programming¶

Framework Comparison¶

Neurenix vs. TensorFlow¶

Neurenix vs. PyTorch¶

Neurenix vs. Scikit-Learn¶

Best Practices¶

Algorithm Selection¶

Population Size¶

Fitness Function Design¶

Hyperparameter Tuning¶

Tutorials¶

Evolving Neural Networks with NEAT¶

Optimizing Neural Network Weights with CMA-ES¶

Evolving Large-Scale Networks with HyperNEAT¶