Metadata-Version: 2.1
Name: mmm-zeta
Version: 0.0.3
Summary: MMM - Pytorch
Home-page: https://github.com/kyegomez/MultiModalMamba
License: MIT
Keywords: artificial intelligence,deep learning,optimizers,Prompt Engineering
Author: Kye Gomez
Author-email: kye@apac.ai
Requires-Python: >=3.6,<4.0
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.9
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Dist: swarms
Requires-Dist: torch (==2.1.2)
Requires-Dist: zetascale (==1.4.0)
Project-URL: Documentation, https://github.com/kyegomez/MultiModalMamba
Project-URL: Repository, https://github.com/kyegomez/MultiModalMamba
Description-Content-Type: text/markdown

[![Multi-Modality](agorabanner.png)](https://discord.gg/qUtxnK2NMf)

# Multi Modal Mamba - [MMM]
A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.


## Install
`pip3 install mmm-zeta`


## Usage
```python
# Import the necessary libraries
import torch 
from torch import nn
from mm_mamba import MultiModalMambaBlock

# Create some random input tensors
x = torch.randn(1, 16, 64)  # Tensor with shape (batch_size, sequence_length, feature_dim)
y = torch.randn(1, 3, 64, 64)  # Tensor with shape (batch_size, num_channels, image_height, image_width)

# Create an instance of the MultiModalMambaBlock model
model = MultiModalMambaBlock(
    dim = 64,  # Dimension of the token embeddings
    depth = 5,  # Number of transformer layers
    dropout = 0.1,  # Dropout probability
    heads = 4,  # Number of attention heads
    d_state = 16,  # Dimension of the state embeddings
    image_size = 64,  # Size of the input image
    patch_size = 16,  # Size of each image patch
    encoder_dim = 64,  # Dimension of the encoder token embeddings
    encoder_depth = 5,  # Number of encoder transformer layers
    encoder_heads = 4  # Number of encoder attention heads
)

# Pass the input tensors through the model
out = model(x, y)

# Print the shape of the output tensor
print(out.shape)

```

# License
MIT




