From f49d3c667417c4cc28c238f3ac3627a2507f514b Mon Sep 17 00:00:00 2001 From: Francis Secada Date: Thu, 23 Jan 2025 17:19:11 -0500 Subject: [PATCH] Updated README file --- README.md | 502 ++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 358 insertions(+), 144 deletions(-) diff --git a/README.md b/README.md index 006a20b..0b4ac89 100644 --- a/README.md +++ b/README.md @@ -1,83 +1,77 @@
- -
- -Project Logo + + # PYGENTIC-AI - -Empowering AI innovation with seamless asynchronous processing. + -license -last-commit -repo-top-language -repo-language-count +license +last-commit +repo-top-language +repo-language-count - +Built with the tools and technologies: - - - -
+Anthropic +Jinja +Redis +SQLAlchemy +TOML +tqdm +Rich +GNU%20Bash +Celery
+FastAPI +Docker +Python +GitHub%20Actions +OpenAI +uv +Pydantic +YAML + +
+ +## โ˜€๏ธ Table of Contents + +1. [โ˜€ ๏ธ Table of Contents](#-table-of-contents) +2. [๐ŸŒž Overview](#-overview) +3. [๐Ÿ”ฅ Features](#-features) +4. [๐ŸŒ… Project Structure](#-project-structure) + 4.1. [๐ŸŒ„ Project Index](#-project-index) +5. [๐Ÿš€ Getting Started](#-getting-started) + 5.1. [๐ŸŒŸ Prerequisites](#-prerequisites) + 5.2. [โšก Installation](#-installation) + 5.3. [๐Ÿ”† Usage](#-usage) + 5.4. [๐ŸŒ  Testing](#-testing) +6. [๐ŸŒป Roadmap](#-roadmap) +7. [๐Ÿค Contributing](#-contributing) +8. [๐Ÿ“œ License](#-license) +9. [โœจ Acknowledgments](#-acknowledgments) --- -## Table of Contents +## ๐ŸŒž Overview + -- [Table of Contents](#table-of-contents) -- [Overview](#overview) -- [Features](#features) -- [Project Structure](#project-structure) - - [Project Index](#project-index) -- [Getting Started](#getting-started) - - [Prerequisites](#prerequisites) - - [Installation](#installation) - - [Usage](#usage) - - [Testing](#testing) -- [Roadmap](#roadmap) -- [Contributing](#contributing) -- [License](#license) -- [Acknowledgments](#acknowledgments) --- -## Overview +## ๐Ÿ”ฅ Features -Pygentic-AI simplifies project setup and backend operations for developers. - -**Why Pygentic-AI?** - -This project automates Python environment setup and dependency management, ensuring a smooth development workflow. The custom logger enhances backend logging efficiency, while the RESTful API server streamlines backend operations. - -- **๐Ÿš€ Automated Python Setup:** Simplify environment configuration and dependency management. -- **๐Ÿ’ก Custom Logger:** Efficient logging with customizable features for backend operations. -- **๐ŸŒ RESTful API Server:** Facilitate seamless backend operations with a robust API server. +โฏ REPLACE-ME --- -## Features - -| | Component | Details | -| :--- | :-------------- | :----------------------------------- | -| โš™๏ธ | **Architecture** | | -| ๐Ÿ”ฉ | **Code Quality** | | -| ๐Ÿ“„ | **Documentation** | | -| ๐Ÿ”Œ | **Integrations** | | -| ๐Ÿงฉ | **Modularity** | | -| ๐Ÿงช | **Testing** | | -| โšก๏ธ | **Performance** | | -| ๐Ÿ›ก๏ธ | **Security** | | -| ๐Ÿ“ฆ | **Dependencies** | | - ---- - -## Project Structure +## ๐ŸŒ… Project Structure ```sh โ””โ”€โ”€ Pygentic-AI/ + โ”œโ”€โ”€ .github + โ”‚ โ””โ”€โ”€ workflows โ”œโ”€โ”€ bin โ”‚ โ”œโ”€โ”€ build.sh โ”‚ โ”œโ”€โ”€ linux_build.sh @@ -90,18 +84,19 @@ This project automates Python environment setup and dependency management, ensur โ”œโ”€โ”€ dev_requirements.txt โ”œโ”€โ”€ docker โ”‚ โ”œโ”€โ”€ celery - โ”‚ โ””โ”€โ”€ pygentic_ai_ms + โ”‚ โ””โ”€โ”€ pygentic_ai โ”œโ”€โ”€ Dockerfile โ”œโ”€โ”€ pyproject.toml โ”œโ”€โ”€ README.md โ”œโ”€โ”€ src โ”‚ โ”œโ”€โ”€ app.py โ”‚ โ”œโ”€โ”€ backend - โ”‚ โ””โ”€โ”€ cworker.py + โ”‚ โ”œโ”€โ”€ cworker.py + โ”‚ โ””โ”€โ”€ frontend โ””โ”€โ”€ uv.lock ``` -### Project Index +### ๐ŸŒ„ Project Index
PYGENTIC-AI/ @@ -120,31 +115,31 @@ This project automates Python environment setup and dependency management, ensur compose.yaml - - Define services and configurations for web and celery_service containers, including image, resources, ports, environment variables, volumes, labels, health checks, and network settings
- Services are deployed with specific memory limits and exposed ports, with web service catering to production and celery_service to staging environments
- Traefik is utilized for routing and SSL termination. + Code>โฏ REPLACE-ME core_requirements.in - Define project dependencies and requirements using the core_requirements.in file to ensure seamless integration and functionality across the codebase architecture. + Code>โฏ REPLACE-ME core_requirements.txt - Generates a compiled list of core requirements for the project, ensuring all necessary dependencies are included for seamless functionality across various modules and components. + Code>โฏ REPLACE-ME dev_requirements.in - - Enhance development environment by managing dependencies efficiently with dev_requirements.in
- This file specifies essential tools like Alembic, Black, and Pre-commit to streamline the development process
- Optimize code formatting and debugging with JupyterLab extensions, ensuring a robust and efficient workflow. + Code>โฏ REPLACE-ME dev_requirements.txt - - Generate a list of development requirements for the project, outlining dependencies and their versions
- The file was auto-generated using uv to compile the dev_requirements.in file
- It includes various packages required for development, such as alembic, fastapi, and jupyter-server
- Each dependency is specified with its version number for accurate package management. + Code>โฏ REPLACE-ME Dockerfile - - Create a Docker image for Pygentic-AI project, setting up necessary environment variables, cloning the project repository, and configuring the workspace
- Install dependencies, create user celery, set up directories, and make scripts executable
- Finally, build and start the Python application within the designated workspace. + Code>โฏ REPLACE-ME pyproject.toml - - Define linting rules and file exclusions for the project structure in the pyproject.toml file
- Ensure code quality by enforcing specific linting standards, excluding unnecessary directories, and setting line length limits
- This configuration enhances code readability and maintainability across the codebase. + Code>โฏ REPLACE-ME @@ -164,19 +159,19 @@ This project automates Python environment setup and dependency management, ensur build.sh - Update and install project dependencies using the provided build script. + Code>โฏ REPLACE-ME linux_build.sh - - Set up a Linux environment for the project by installing essential dependencies and tools
- Ensure a smooth development experience by preparing the system with required packages like build tools, Python 3.13, PostgreSQL, and more
- This script streamlines the initial setup process, enabling developers to focus on building the project without worrying about environment configuration. + Code>โฏ REPLACE-ME python_build.sh - - Automates Python environment setup and dependency management for the project
- Creates a virtual environment, installs necessary packages, and syncs core and development requirements using pip-tools
- Facilitates streamlined development workflow by ensuring consistent package versions across environments. + Code>โฏ REPLACE-ME start.sh - - Launches the application using Gunicorn with specified configurations, such as the number of workers, timeout, and port
- The script activates the virtual environment and starts the server to handle incoming requests. + Code>โฏ REPLACE-ME @@ -196,11 +191,11 @@ This project automates Python environment setup and dependency management, ensur app.py - - Define exception handlers and mount static files for the FastAPI app using the provided code
- Handle validation errors and custom exceptions, logging details and returning appropriate responses
- Serve static files from the specified directory for the frontend. + Code>โฏ REPLACE-ME cworker.py - - Improve concurrency by managing worker threads efficiently
- This code file in src\cworker.py orchestrates thread creation and execution within the projects architecture. + Code>โฏ REPLACE-ME @@ -218,11 +213,11 @@ This project automates Python environment setup and dependency management, ensur logger.py - - Implement a custom logger using Loguru for efficient logging in the backend
- The logger supports various log levels and customization options, enhancing the logging experience
- It includes features like log rotation, retention, and different log formats
- The code ensures robust logging functionality for the project. + Code>โฏ REPLACE-ME utils.py - - Define utility functions to retrieve database URLs based on environment and fetch values from environment variables or configuration files
- The get_db_url function constructs a database URL for different environments, while get_val retrieves values with fallback options
- These functions enhance flexibility and maintainability in managing configurations and environment variables. + Code>โฏ REPLACE-ME @@ -240,23 +235,23 @@ This project automates Python environment setup and dependency management, ensur consts.py - - Define the AI_MODEL and default_system_prompt for the GPT-4o AI assistant in the consts.py file
- The AI_MODEL specifies the model used, while the default_system_prompt outlines the AIs function of generating SWOT analyses. + Code>โฏ REPLACE-ME core.py - - Define a SQLModel and Agent Dependencies for SWOT Analysis, creating a SwotAgent with specified parameters for AI model, system prompt, and retries
- This code file in the core module plays a crucial role in structuring and managing SWOT analysis responses within the projects architecture. + Code>โฏ REPLACE-ME main.py - Implement core functionality for backend services in the main.py file. + Code>โฏ REPLACE-ME tools.py - - Describe how the tools.py file in src\backend\core facilitates fetching website content, analyzing competition using the Gemini model, and obtaining insights from Reddit
- The functions within this file leverage various libraries and APIs to perform these tasks efficiently. + Code>โฏ REPLACE-ME utils.py - - Define a function to report tool usage and results within the projects core architecture
- The function checks and updates tool usage history, notifying an update function if available
- This contributes to tracking tool usage and ensuring status updates are communicated effectively. + Code>โฏ REPLACE-ME @@ -276,27 +271,27 @@ This project automates Python environment setup and dependency management, ensur base.py - - Generate base SQLModel class for project, addressing Pydantic V2.5 issue with __pydantic_extra__ attribute
- Inherits from SQLModel and sets necessary attributes
- Integrated with project metadata and configured for eager defaults. + Code>โฏ REPLACE-ME consts.py - Define database constants for backend operations in the project structure. + Code>โฏ REPLACE-ME core.py - Manage core database operations for the backend system, ensuring efficient data handling and storage. + Code>โฏ REPLACE-ME db.py - - Create and manage async SQLA sessions, handle database operations, and ensure database schema creation
- Includes functions for checking database existence, creating databases, and setting the current schema
- Provides utilities for creating database engines and managing database connections
- Mainly focuses on database setup and interaction for the project. + Code>โฏ REPLACE-ME main.py - Implement database connection and query functions to manage data persistence for the backend services. + Code>โฏ REPLACE-ME utils.py - - Enhances database operations by providing utility functions
- Facilitates seamless interaction with the database layer. + Code>โฏ REPLACE-ME @@ -316,23 +311,23 @@ This project automates Python environment setup and dependency management, ensur consts.py - Define and store constant values used throughout the backend server architecture. + Code>โฏ REPLACE-ME core.py - Implement core server functionality to handle incoming requests and manage data operations within the backend architecture. + Code>โฏ REPLACE-ME main.py - - Implement a RESTful API server in Python to handle backend operations for the project
- The main.py file serves as the entry point for the server, orchestrating requests and responses
- It plays a crucial role in managing the communication between the frontend and backend components of the application. + Code>โฏ REPLACE-ME router.py - Define API routes and request handling logic for the backend server. + Code>โฏ REPLACE-ME utils.py - - Enhances server functionality by providing utility functions
- Improves codebase architecture by centralizing common operations
- Facilitates streamlined development and maintenance processes
- Promotes code reusability and efficiency within the backend server module. + Code>โฏ REPLACE-ME @@ -352,35 +347,35 @@ This project automates Python environment setup and dependency management, ensur backend_options.py - - Generate SQL database connection URLs based on provided configurations for different environments using accepted dialects, including PostgreSQL and SQLite
- Import these configurations into the appropriate settings file to integrate with the Django Database settings. + Code>โฏ REPLACE-ME base.py - Define project settings including backend and frontend directories, database URL, and debug mode in the base settings file. + Code>โฏ REPLACE-ME consts.py - Define database dialects and set secret key for project configuration. + Code>โฏ REPLACE-ME core.py - - Enhances core settings functionality by providing centralized configuration management
- Facilitates seamless access and modification of key system parameters
- Improves maintainability and scalability of the project by consolidating settings logic. + Code>โฏ REPLACE-ME dev.py - Define development settings for backend with local database configuration and debugging enabled. + Code>โฏ REPLACE-ME main.py - - Enhances backend settings functionality by managing configurations efficiently
- Facilitates seamless customization and optimization of settings across the codebase architecture. + Code>โฏ REPLACE-ME prod.py - - Define production settings for the backend, including database configuration and debugging options
- Extends base settings to inherit common configurations
- Centralizes cloud database settings for easy management and maintenance. + Code>โฏ REPLACE-ME utils.py - - Enhance backend settings functionality by providing utility functions for the codebase architecture
- The utils.py file in the src\backend\settings directory plays a crucial role in enabling streamlined operations and improved performance within the project structure. + Code>โฏ REPLACE-ME @@ -400,29 +395,241 @@ This project automates Python environment setup and dependency management, ensur consts.py - - Define constants and data structures for tracking analysis progress and results in the backend of the site architecture
- Includes messages for analysis status, sets for running tasks, and dictionaries for storing status and results. + Code>โฏ REPLACE-ME core.py - Implement core functionality for the site backend, facilitating key operations within the project architecture. + Code>โฏ REPLACE-ME main.py - Improve site performance by caching API responses in the main.py file. + Code>โฏ REPLACE-ME router.py - - Implement a backend API router for a SWOT analysis tool
- The router handles URL analysis requests, updates status messages, and provides analysis results
- It manages session IDs, progress tracking, and result storage
- The code integrates with FastAPI, Starlette, and Jinja2 for web functionality. + Code>โฏ REPLACE-ME utils.py - - Enhances backend functionality by providing utility functions for the site
- This code file in the backend architecture aids in streamlining operations and improving overall performance. + Code>โฏ REPLACE-ME
+ +
+ frontend +
+
+ โฆฟ src.frontend + +
+ templates +
+
+ โฆฟ src.frontend.templates + + + + + + + + + + + + + + + + + + + +
File NameSummary
home.htmlCode>โฏ REPLACE-ME
result.htmlCode>โฏ REPLACE-ME
status.htmlCode>โฏ REPLACE-ME
+ +
+ components +
+
+ โฆฟ src.frontend.templates.components + +
+ forms +
+
+ โฆฟ src.frontend.templates.components.forms + + + + + + + + + + + + + + + +
File NameSummary
Form.jinjaCode>โฏ REPLACE-ME
Search.jinjaCode>โฏ REPLACE-ME
+
+
+ +
+ main +
+
+ โฆฟ src.frontend.templates.components.main + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
File NameSummary
base.htmlCode>โฏ REPLACE-ME
Base.jinjaCode>โฏ REPLACE-ME
footer.htmlCode>โฏ REPLACE-ME
Footer.jinjaCode>โฏ REPLACE-ME
header.htmlCode>โฏ REPLACE-ME
Header.jinjaCode>โฏ REPLACE-ME
nav.htmlCode>โฏ REPLACE-ME
Nav.jinjaCode>โฏ REPLACE-ME
Scripts.jinjaCode>โฏ REPLACE-ME
Stylesheets.jinjaCode>โฏ REPLACE-ME
style_sheets.htmlCode>โฏ REPLACE-ME
+
+
+ +
+ snippets +
+
+ โฆฟ src.frontend.templates.components.snippets + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
File NameSummary
Css.jinjaCode>โฏ REPLACE-ME
js.htmlCode>โฏ REPLACE-ME
Js.jinjaCode>โฏ REPLACE-ME
NavbarBrand.jinjaCode>โฏ REPLACE-ME
NavbarMenu.jinjaCode>โฏ REPLACE-ME
Result.jinjaCode>โฏ REPLACE-ME
ResultEntry.jinjaCode>โฏ REPLACE-ME
Spinner.jinjaCode>โฏ REPLACE-ME
StatusResult.jinjaCode>โฏ REPLACE-ME
+
+
+
+
+
+
+
+
+ + + +
+ .github +
+
+ โฆฟ .github + +
+ workflows +
+
+ โฆฟ .github.workflows + + + + + + + + + + + + + + + +
File NameSummary
bandit.ymlCode>โฏ REPLACE-ME
docker-image.ymlCode>โฏ REPLACE-ME
+
+
@@ -446,17 +653,17 @@ This project automates Python environment setup and dependency management, ensur start.sh - - Initiates and manages Celery workers, beat scheduler, and Flower for the projects asynchronous task processing
- Handles worker availability checks and starts necessary services within the specified project directory. + Code>โฏ REPLACE-ME - +
- pygentic_ai_ms + pygentic_ai
- โฆฟ docker.pygentic_ai_ms + โฆฟ docker.pygentic_ai @@ -465,16 +672,16 @@ This project automates Python environment setup and dependency management, ensur - - + + - - + + - - + +
build.sh- Install necessary dependencies for the ranked jobs microservice in the Docker container
- The script sets up essential tools like Python, PostgreSQL, and Git, ensuring a robust environment for the microservice to run smoothly.		build.shCode>โฏ REPLACE-ME
python_build.sh- Automates Python environment setup and dependency management for the ranked jobs microservice in the Docker architecture
- Sets up virtual environment, compiles requirements, and syncs dependencies for smooth execution.		python_build.shCode>โฏ REPLACE-ME
python_start.sh- Launches the Python application using Gunicorn with specified configurations like the number of workers, timeout, and port
- The script activates the virtual environment and starts the server to handle incoming requests.		python_start.shCode>โฏ REPLACE-ME
@@ -485,9 +692,9 @@ This project automates Python environment setup and dependency management, ensur --- -## Getting Started +## ๐Ÿš€ Getting Started -### Prerequisites +### ๐ŸŒŸ Prerequisites This project requires the following dependencies: @@ -495,7 +702,7 @@ This project requires the following dependencies: - **Package Manager:** Pip, Uv - **Container Runtime:** Docker -### Installation +### โšก Installation Build Pygentic-AI from the source and intsall dependencies: @@ -514,40 +721,47 @@ Build Pygentic-AI from the source and intsall dependencies: 3. **Install the dependencies:** - - - - + + + + - **Using [docker](https://www.docker.com/):** +**Using [docker](https://www.docker.com/):** ```sh โฏ docker build -t fsecada01/Pygentic-AI . ``` - - - - - - **Using [pip](https://pypi.org/project/pip/):** + + + + + + +**Using [pip](https://pypi.org/project/pip/):** ```sh - โฏ pip install -r core_requirements.in, core_requirements.txt, dev_requirements.in, dev_requirements.txt + โฏ pip install -r core_requirements.txt, dev_requirements.txt ``` - - - - - +If this fails due to platform-specific issues, try this instead: - **Using [uv](https://docs.astral.sh/uv/):** + ```sh + โฏ pip install -r core_requirements.in, dev_requirements.in + ``` + + + + + + + +**Using [uv](https://docs.astral.sh/uv/):** ```sh โฏ uv sync --all-extras --dev ``` -### Usage +### ๐Ÿ”† Usage Run the project with: @@ -564,7 +778,7 @@ python {entrypoint} uv run python {entrypoint} ``` -### Testing +### ๐ŸŒ  Testing Pygentic-ai uses the {__test_framework__} test framework. Run the test suite with: @@ -579,15 +793,15 @@ uv run pytest tests/ --- -## Roadmap +## ๐ŸŒป Roadmap -- [X] **`Task 1`**: Implement feature one. -- [ ] **`Task 2`**: Implement feature two. -- [ ] **`Task 3`**: Implement feature three. +- [X] **`Task 1`**: Bootstrap a minimal application build +- [ ] **`Task 2`**: Implement DB Backend with PostgreSQL +- [ ] **`Task 3`**: Integrate user auth and group controls --- -## Contributing +## ๐Ÿค Contributing - **๐Ÿ’ฌ [Join the Discussions](https://github.com/fsecada01/Pygentic-AI/discussions)**: Share your insights, provide feedback, or ask questions. - **๐Ÿ› [Report Issues](https://github.com/fsecada01/Pygentic-AI/issues)**: Submit bugs found or log feature requests for the `Pygentic-AI` project. @@ -630,13 +844,13 @@ uv run pytest tests/ --- -## License +## ๐Ÿ“œ License Pygentic-ai is protected under the [LICENSE](https://choosealicense.com/licenses) License. For more details, refer to the [LICENSE](https://choosealicense.com/licenses/) file. --- -## Acknowledgments +## โœจ Acknowledgments - Credit `contributors`, `inspiration`, `references`, etc.