Bandit Problems

mutable struct MyEpsilonSamplingBanditModel <: AbstractSamplingModel

The MyEpsilonSamplingBanditModel mutable struct represents a multi-armed bandit model that uses epsilon-sampling for exploration.

Required fields

• α::Array{Float64,1}: A vector holding the number of successful pulls for each arm. Each element in the vector represents the number of successful pulls for a specific arm.
• β::Array{Float64,1}: A vector holding the number of unsuccessful pulls for each arm. Each element in the vector represents the number of unsuccessful pulls for a specific arm.
• K::Int64: The number of arms in the bandit model
• ϵ::Float64: The exploration parameter. A value of 0.0 indicates no exploration, and a value of 1.0 indicates full exploration.

function build(modeltype::Type{MyEpsilonSamplingBanditModel}, data::NamedTuple) -> MyEpsilonSamplingBanditModel

This build method constructs an instance of the MyEpsilonSamplingBanditModel type using the data in a NamedTuple.

Arguments

• modeltype::Type{MyEpsilonSamplingBanditModel}: The type of model to build, in this case, the MyEpsilonSamplingBanditModel type.
• data::NamedTuple: The data to use to build the model.

The data::NamedTuple must contain the following keys:

• α::Array{Float64,1}: A vector holding the number of successful pulls for each arm. Each element in the vector represents the number of successful pulls for a specific arm.
• β::Array{Float64,1}: A vector holding the number of unsuccessful pulls for each arm. Each element in the vector represents the number of unsuccessful pulls for a specific arm.
• K::Int64: The number of arms in the bandit model
• ϵ::Float64: The exploration parameter. A value of 0.0 indicates no exploration, and a value of 1.0 indicates full exploration.

mutable struct MyTickerPickerWorldModel <: AbstractWorldModel

The MyTickerPickerWorldModel mutable struct represents a world model for a ticker picker problem.

Required fields

• tickers::Array{String,1}: An array of ticker symbols that we explore
• data::Dict{String, DataFrame}: A dictionary that holds the data for each ticker symbol
• risk_free_rate::Float64: The risk-free rate of return in the world (assumed constant)
• world::Function: A function that represents the world model. The function takes an action a, data about the world, and returns the reward for taking action a.
• Δt::Float64: The time step size in the world model
• buffersize::Int64: The size of the buffer used in the world model

function build(modeltype::Type{MyTickerPickerWorldModel}, data::NamedTuple) -> MyTickerPickerWorldModel

This build method constructs an instance of the MyTickerPickerWorldModel type using the data in a NamedTuple.

Arguments

• modeltype::Type{MyTickerPickerWorldModel}: The type of model to build, in this case, the MyTickerPickerWorldModel type.
• data::NamedTuple: The data to use to build the model.

The data::NamedTuple must contain the following keys:

• tickers::Array{String,1}: An array of ticker symbols that we explore
• data::Dict{String, DataFrame}: A dictionary that holds the data for each ticker symbol
• horizon::Int64: The number of days to look ahead for the ticker picker
• buffersize::Int64: The size of the buffer for storing the data

function sample(model::MyEpsilonSamplingBanditModel, world::AbstractWorldModel)::Dict{Int64, Array{Beta,1}}

This function solved the ticker picker problem as a bandit model using an epsilon-greedy strategy. The function takes two arguments: a bandit model and a world model. The bandit model is a custom type, MyEpsilonSamplingBanditModel, which contains the parameters of the bandit model. The world model is a custom type, MyTickerPickerWorldModel, which contains the parameters of the world model. The function returns a dictionary where the keys are integers (time steps) and the values are arrays of Beta distributions.

Arguments

• model::MyEpsilonSamplingBanditModel: An instance of the MyEpsilonSamplingBanditModel that defines the bandit model parameters.
• world::AbstractWorldModel: An instance of the world model that defines the world model parameters.
• horizon::Int64 = 1: The number of trials to sample. Default is 1 (single trial).

Returns

• Dict{Int64, Array{Beta,1}}: A dictionary where the keys are integers (trials) and the values are arrays of Beta distributions (ticker preferences).

function preference(beta::Array{Beta,1}, tickers::Array{String,1}) -> Array{Int64,1}

This function computes the preference of each action based on the mean of the Beta distribution.

Arguments

• beta::Array{Beta,1}: An array of Beta distributions that represent the actions (or preferences)
• tickers::Array{String,1}: An array of strings that represent the tickers (or actions)

Returns

• Array{Int64,1}: An array of integers that represent the preference of each action based on the mean of the Beta distribution.

mutable struct MyTickerPickerRiskAwareWorldModel <: AbstractWorldModel

The MyTickerPickerRiskAwareWorldModel mutable struct represents a world model for a ticker picker problem that is risk-aware.

Required fields

• tickers::Array{String,1}: An array of ticker symbols that we explore
• data::Dict{String, DataFrame}: A dictionary that holds the price data for each ticker symbol
• risk_free_rate::Float64: The risk-free rate of return in the world (assumed constant)
• world::Function: A function that represents the world model. The function takes an action a, data about the world, and returns the reward r for taking action a.
• Δt::Float64: The time step size in the world model
• buffersize::Int64: The size of the buffer used in the world model
• risk::Dict{String, Float64}: A dictionary that holds the risk measure for each ticker symbol

function build(modeltype::Type{MyTickerPickerRiskAwareWorldModel}, data::NamedTuple) -> MyTickerPickerRiskAwareWorldModel

This build method constructs an instance of the MyTickerPickerRiskAwareWorldModel type using the data in a NamedTuple.

Arguments

• modeltype::Type{MyTickerPickerRiskAwareWorldModel}: The type of model to build, in this case, the MyTickerPickerRiskAwareWorldModel type.
• data::NamedTuple: The data to use to build the model.

The data::NamedTuple must contain the following keys:

• tickers::Array{String,1}: An array of ticker symbols that we explore
• data::Dict{String, DataFrame}: A dictionary that holds the price data for each ticker symbol
• risk_free_rate::Float64: The risk-free rate of return in the world (assumed constant)
• world::Function: A function that represents the world model. The function takes an action a, data about the world, and returns the reward r for taking action a.
• Δt::Float64: The time step size in the world model
• buffersize::Int64: The size of the buffer used in the world model
• risk::Dict{String, Float64}: A dictionary that holds the risk measure for each ticker symbol