Lade till requests

.gitignore

@@ -160,3 +160,6 @@ cython_debug/
 #  option (not recommended) you can uncomment the following to ignore the entire idea folder.
 #.idea/
 
+# Exclude venv from smartassist
+smartassist/smartassist_dev_venv
+.DS_Store

.prompts/explain.prompt

@@ -0,0 +1,21 @@
+# This is "explain.prompt", a slash command to explain code
+# It is used to define and reuse prompts within Continue
+# Continue will automatically create a slash command for each prompt in the .prompts folder
+# To learn more, see the full .prompt file reference: https://docs.continue.dev/walkthroughs/prompt-files
+temperature: 0.3
+---
+<system>
+You are an expert programmer
+</system>
+
+{{{ input }}}
+
+Please analyze the source code snippet above and provide a detailed explanation of its functionality. 
+
+Input: Clearly identify where the code takes input from (e.g., user input, file, database, API call). Specify the format of this input (e.g., text string, numerical values, JSON object).
+Processing: Describe step-by-step how the code processes the input. Explain the purpose of each key function, loop, or conditional statement. Use clear and concise language, avoiding jargon where possible.
+Output: Specify what the code produces as output (e.g., printed text, modified file, database update, API response). Describe the format and content of this output.
+
+Invocation Context: Who would typically use this code snippet? What is its intended purpose or application? Provide examples of real-world scenarios where this code might be employed.
+
+If you need to provide more context, please include it in your explanation.

smartassist/config/smartassist.yaml

@@ -0,0 +1,41 @@
+# Backend Configuration
+backend:
+  url: "http://localhost:5004"
+  api: "/api/chat"
+
+preferred_ep: "Ollama-WARA"
+
+endpoints: 
+  - model: "AUTODETECT"
+    title: "Ollama-local" # Must be a unique identifier 
+    url: "http://localhost:11434"
+    provider: "ollama"
+  # - model: "AUTODETECT"  
+  - model: "llava:13b"
+    title: "Ollama-WARA" # Must be a unique identifier 
+    url: "https://ollama-test.wara-ops.org"
+    requestOptions:
+      headers:
+        Authorization: "${OLLAMA_API_KEY}" # on MacOS: echo "Authorization: Basic $(echo -n 'user:password' | gbase64 -w 0)"
+    provider: "ollama"
+
+
+
+# Logging – comment out the whole section for default level which is INFO
+logging:
+  level: DEBUG  # Set log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+  # level: INFO  # Set log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
+
+# Additional Configuration Options (Optional – ignored for now)
+
+# Cache Settings (Optional)
+cache:
+  enabled: True
+  timeout: 60 # Seconds 
+
+test:
+  test_key_1: "ett"
+  test_key_2: "ett"
+  test_key_3: "${THIS_IS_A_DUMMY_VAR}"
+
+another_test: "Syns detta?"

smartassist/readme.md

@@ -0,0 +1,69 @@
+ propose a python project that involves using using ollama for using the power of LLM
+ 
+ 
+ 
+ Certainly! Let's design a Python project that leverages the capabilities of an Open Language Learning Model Architecture (OLLAMA) to create a versatile and powerful application. Our project will be named "SmartAssist", which is an intelligent assistant powered by a large language model (LLM). The main goal of SmartAssist is to provide users with personalized assistance, recommendations, and information retrieval through natural language processing and machine learning techniques.
+
+### Project Overview:
+
+**Name:** SmartAssist
+
+**Objective:** Develop a versatile AI-powered assistant that can interact with users in natural language and perform tasks such as answering questions, providing recommendations, summarizing texts, and more.
+
+### Features:
+
+1. **Natural Language Processing (NLP):** Utilize OLLAMA to understand user queries and generate appropriate responses.
+2. **Task Automation:** Enable the assistant to perform various automated tasks based on user requests.
+3. **Integration with External APIs:** Fetch real-time data from external sources to provide up-to-date information.
+4. **User Preferences:** Allow users to customize their interaction experience by setting preferences or providing feedback.
+5. **Multi-lingual Support:** Expand the assistant's capabilities to support multiple languages if OLLAMA offers this feature.
+6. **Continuous Learning:** Use machine learning algorithms to continuously improve the model's performance based on user interactions and feedback.
+7. **Cross-platform Compatibility:** Develop a web app, mobile app, or desktop application that can be accessed from various devices.
+
+### Technical Architecture:
+
+1. **Backend:** Python with Flask or FastAPI for creating RESTful APIs to handle requests from the frontend and interact with OLLAMA.
+2. **OLLAMA Integration:** Use a library or API provided by OLLAMA to interface with the large language model.
+3. **Frontend:** React Native (for mobile), Vue.js (or Angular) for web, or Swift/Kotlin (for iOS and Android).
+4. **Database:** SQLite for storing user preferences, interaction logs, and other data required by the application.
+5. **Continuous Integration/Deployment:** Implement CI/CD pipelines using GitHub Actions or GitLab CI to automate testing and deployment processes.
+
+### Implementation Steps:
+
+1. **Setup Environment:** Install necessary libraries such as Flask, SQLite, and OLLAMA-related dependencies.
+2. **API Endpoints:** Create API endpoints for user interactions (GET/POST requests).
+3. **OLLAMA Integration:** Implement a function to interact with OLLAMA using its API or library, allowing the assistant to generate responses based on input queries.
+4. **Frontend Development:** Develop responsive and interactive interfaces for users to engage with SmartAssist.
+5. **Testing:** Perform unit testing, integration testing, and user acceptance testing to ensure the application works as expected.
+6. **Deployment:** Deploy the application using cloud services like AWS, Azure, or Google Cloud, ensuring scalability and high availability.
+7. **Maintenance and Updates:** Regularly update OLLAMA library versions and add new features based on feedback and emerging technologies.
+
+### Example Code Snippet:
+
+```python
+# Example function to interact with OLLAMA
+from ollama import OllamaClient
+
+def get_response(user_query):
+    client = OllamaClient()
+    response = client.generate_response(user_query)
+    return response
+
+# Flask endpoint for user interaction
+from flask import Flask, request, jsonify
+app = Flask(__name__)
+
+@app.route('/smartassist', methods=['POST'])
+def smartassist():
+    data = request.json
+    user_query = data['query']
+    response = get_response(user_query)
+    return jsonify({"response": response})
+
+if __name__ == '__main__':
+    app.run(debug=True)
+```
+
+### Conclusion:
+
+By developing SmartAssist, we will create a versatile and intelligent assistant that can handle various tasks through natural language processing, providing users with personalized assistance across multiple platforms. This project not only showcases the power of OLLAMA but also highlights the importance of continuous learning and adaptation in AI-driven applications.

smartassist/requirements_dev.txt

@@ -0,0 +1,10 @@
+# A requirements file for the project.
+#
+# Run 'pip install -r smartassist/requirements_dev.txt' to install all required packages.
+
+
+ollama
+Flask
+flask_cors
+requests
+PyYAML

smartassist/src/backend.py

@@ -0,0 +1,318 @@
+
+# Import the necessary functions from ollama, Flask, requests, threading
+from ollama import Client
+from flask import Flask, request, jsonify, send_from_directory, render_template, session, make_response, Response
+from flask_cors import CORS, cross_origin # CORS stands for Cross-Origin Resource Sharing. This is necessary to allow the frontend to make requests to our backend.
+import requests
+import json
+import logging
+import os
+import utils
+from utils import GlobalState
+from pathlib import Path
+
+# Create a logger for this module
+global_state = GlobalState() # Import the singleton that holds global states (e.g., logger)
+logger = global_state.get_logger(__name__)  # Logger for this module, inherit properties of the root logger
+
+
+# Find out the path to current directory according to the Python interpreter (venv)
+logger.debug("Current working directory: %s", os.getcwd())
+
+# Initialize a Flask application
+app = Flask(__name__)
+app.config['STATIC_FOLDER'] = 'static' # Adjust if needed 
+
+# Increase the maximum cookie size
+app.config['SESSION_COOKIE_SAMESITE'] = 'Lax'
+app.config['SESSION_COOKIE_SIZE_LIMIT'] = 4096 * 2  # Allow up to 8KB cookies
+
+# Set the secret key for session management
+secret_key = os.urandom(24)
+app.config['SECRET_KEY'] = secret_key # When do I need this. How is it retained between sessions?
+
+# Optionally set other configuration options
+app.config['SESSION_PERMANENT'] = False  # Session will expire after each request
+app.config['SESSION_TYPE'] = 'filesystem'  # Store sessions on the filesystem
+
+
+logger.debug("flask app template folder: %s", app.template_folder) 
+
+@app.route('/')
+def index() -> Response:
+    """
+    This route serves index.html to connecting clients.
+
+    Initializes a new chat session by clearing the chat history in the session object.
+    Retrieves environment variables for the backend API endpoint, host URL of LLMs, and the selected LLM model.
+    Reads the client HTML template from file and passes it to the index.html template along with other necessary parameters.
+
+    Returns:
+        Response: A Flask response containing the rendered index.html template.
+    """
+    session['chat_history'] = []   # The session object (actually, a dictonary) holds the chat session
+    logger.debug("Entering route '/'")
+    api_endpoint = global_state.get_backend_api_ep()  # Retrieve the environment variable 
+    host_url = global_state.get_host_url()
+    use_model = global_state.get_llm()
+    logger.debug("Backend API endpoint:\t%s", api_endpoint)
+    logger.debug("Host of LLMs:\t\t%s", host_url)
+    logger.debug("LLM to use:\t\t\t%s", use_model)
+    with open('smartassist/src/html/client.html', 'r') as f:
+        client_html = f.read()
+    # logger.debug("Client HTML (first few characters): %s", client_html[:50]) # Print to see if it's loading
+    # logger.debug("Client HTML (all characters): %s", client_html) # Print to see if it's loading
+
+    return render_template('index.html', api_endpoint=api_endpoint, use_model = use_model, client_content=client_html)
+
+@app.route('/set_session')
+def set_session():
+    resp = make_response()
+    resp.set_cookie('session', 'some-value', samesite='None', secure=True) # Add SameSite attribute here
+    return resp
+
+# @app.route('/profile')
+# def profile():
+#     # Retrieve data from the session
+#     user_id = session.get('user_id')
+
+#     if user_id:
+#         return f'User ID: {user_id}'
+#     else:
+#         return 'No user ID found'
+    
+
+
+@app.route('/<path:filename>')
+def serve_static(filename: str | Path) -> Response:
+    """
+    Serves a static file from the application's static folder.
+
+    Args:
+        filename (str or os.PathLike[str]): The path to the static file, relative to the STATIC_FOLDER directory.
+
+    Returns:
+        Response: A Flask response containing the contents of the static file.
+    """
+    return send_from_directory(app.config['STATIC_FOLDER'], filename)
+
+
+
+# CORS(app, resources={
+#     r"/api/chat": {
+#         "origins": "*",
+#         "headers": ["Origin", "Content-Type", "Authorization"],
+#     }
+# })
+
+# CORS(app, resources={
+#     r"/api/chat": {
+#         "origins": "*"    
+#     }
+# })
+    
+@app.route('/api/tags', methods=['GET'])
+def get_tags(url: str = "http://localhost:11434/api/tags", headers: dict = None) -> dict:
+    """
+    Retrieves a list of available models from a server.
+
+    Args:
+        url (str): The URL of the server to query. Defaults to http://localhost:11434/api/tags.
+        headers (dict, optional): A dictionary of HTTP headers to include in the request. Defaults to None.
+
+    Returns:
+        dict: A JSON response containing a list of available models, or an error message if the request fails.
+
+    Raises:
+        requests.exceptions.RequestException: If there is a problem with the request.
+    """
+    try:
+        logger.debug(f"url: {url} headers: {headers}")
+        response = requests.get(url, headers=headers)
+        return response.json()
+    except requests.exceptions.RequestException as e:
+        logger.error("Request Exception: %s", str(e))
+        return {'error': 'Failed to process request'}
+
+
+
+
+@app.route('/api/chat', methods=['POST'])
+def chat() -> dict[str, any]:
+    """
+    Handles chat functionality by sending a query to an LLM server and 
+    returning the response.
+
+    This endpoint expects a JSON payload with the following structure:
+    {
+        'query': str,
+        'url_server': str (optional),
+        'model': str (optional)
+    }
+
+    :return: A dictionary containing the LLM's response
+    """
+    # Get the message from the JSON in the request body
+    data = request.get_json()
+    message = data.get('query')
+    url_server = data.get('url_server', global_state.get_host_url())  # Use provided URL or current if not provided
+    # url_server = data.get('url_server', "https://ollama-test.wara-ops.org/api/generate")  # Use provided URL or default
+    model = data.get('model', global_state.get_llm())  # Use provided model or current if not provided
+    
+    # Get chat history from session storage (e.g., a dictionary)
+    chat_history = session.get('chat_history', [])
+    
+    # Add the new message to the chat history
+    chat_history.append({'role': 'user', 'message': message})
+    
+    # Update the session with the new chat history
+    session['chat_history'] = chat_history
+    
+    # Create the data dictionary with chat history
+    data_to_send = {
+        "model": model,
+        'prompt': '\n'.join([f"{item['role']}: {item['message']}" for item in chat_history]),
+        "stream": False           
+    }
+    url = url_server
+    headers = get_auth_headers(url)
+
+    logger.debug(f"Sending request to:\n\turl:\t{url}\n\tmodel:\t{model}")
+    try:
+        url = url + "/api/generate"
+        logger.debug(f"url: {url} headers: {headers}")
+        response = requests.post(url,
+                                 headers=headers, 
+                                 data=json.dumps(data_to_send))
+        response.raise_for_status()  # Raise an exception for bad status codes
+        llm_response = response.json()['response']   # Assuming the LLM's response is under 'response' key
+        chat_history.append({'role': 'assistant', 'message': llm_response})  # Add assistant response to chat history
+        logger.debug(f"Chat History: {chat_history}")
+        return response.json()
+    except requests.exceptions.RequestException as e:
+        logger.error("Request Exception: %s", str(e))        
+        return jsonify({'error': 'Failed to process request'}), 500
+    except json.JSONDecodeError as e:
+        logger.error("JSON Decode Error: %s", str(e))   # Corresponds to print(f"JSON Decode Error: {e}")             
+        return jsonify({'error': 'Invalid JSON response from server'}), 500
+
+
+@app.route('/api/endpoints', methods=['GET'])
+def get_endpoints() -> str:
+    """
+    Returns a list of available endpoints with their corresponding LLMs.
+
+    This endpoint fetches all endpoints and their associated LLMs from the global state,
+    then returns them as a JSON response.
+
+    :return: A JSON string representing a dictionary containing a list of dictionaries,
+             each representing an endpoint title and supported LLM.
+    """
+    endpoints = []  # List of dictionaries, each of which contains {'title': 'title1', 'llm': 'llm1'}
+    eps = global_state.get_endpoints()
+    for ep in eps:
+        llms = global_state.get_list_of_available_llms(ep)
+        for llm in llms:
+            endpoints.append({'title': ep.get('title'), 'llm': llm})
+    return jsonify(endpoints)
+
+@app.route('/api/select_endpoint_llm', methods=['POST'])
+def select_endpoint_llm() -> Response:
+    """
+    Selects the endpoint associated with the tuple (title, LLM) from the request body.
+
+    Request Body:
+        - title: str - The title of the endpoint to select.
+        - llm: str - The LLM to set.
+
+    Returns:
+        A JSON response indicating whether the endpoint and LLM were selected successfully.
+
+    Raises:
+        ValueError: If there is not exactly one endpoint with the specified title.
+    """
+    data = request.get_json()
+    title = data['title']
+    llm = data['llm']
+
+    endpoints = global_state.get_endpoints_with_key_value('title', title)
+    if len(endpoints) != 1:
+        raise ValueError(f"Expected exactly one endpoint with title '{title}', found {len(endpoints)}")
+
+    # Reset the session
+    if (title != global_state.get_host_title()) or (llm != global_state.get_llm()): # A change in setting 
+        session.clear()
+        logger.debug('Session cleared due to changed endpoint or changed LLM')
+        global_state.set_host_url(endpoints[0]['url'])
+        global_state.set_llm(llm)
+        logger.debug(f"Updated to host url {endpoints[0]['url']} and LLM {llm}")
+        return jsonify({'message': 'New endpoint and/or LLM detected, settings were changed successfully'})
+    else:
+        return jsonify({'message': 'Endpoint and LLM are untouched'})
+
+
+@app.route('/smartassist', methods=["POST"])
+def smartassist():
+    # Extract the query from the incoming JSON data
+    data = request.json
+    user_query = data['query']
+
+    # Get the response from the OLLAMA API based on the user's query
+    # NOTE: Should we append message history here? Maybe interact with SQLlite?
+    response = get_response(user_query)
+
+    # Return the response as a JSON object in the HTTP response
+    return jsonify({"response": response})
+
+def get_response(user_query):
+    client = Client()   # Create a client object for interacting with OLLAMA API
+    response = client.generate_response(user_query)   # Generate and retrieve the response based on user's query
+    return response     
+
+def get_auth_headers(url: str) -> dict:
+    """
+    Returns authentication headers for a given URL.
+
+    This function checks if an endpoint with the provided URL exists in the global state,
+    and returns the corresponding authentication headers. If no such endpoint is found,
+    it returns a default header.
+    """
+    # TODO: The full operation should only have to run when changing to new endpoint.
+
+    # Set default header
+    headers = {
+        "Content-Type": "application/json",
+    }
+
+    found_endpoint = False
+    endpoints = global_state.get_endpoints()
+    for endpoint in endpoints:
+        if endpoint["url"] == url:  # Look for endpoint with this URL
+            found_endpoint = True
+            #if endpoint["provider"] == "ollama":  # Currently only supporting ollama servers - not needed if API the same
+            if "requestOptions" in endpoint:  # Check if authentication is needed
+                headers.update({
+                    "Authorization": endpoint["requestOptions"]["headers"]["Authorization"]
+                })
+    if not found_endpoint:
+        logger.debug(f"Host {url} not found")
+
+    return headers
+
+def run_flask(fport=5005):
+    """
+    Starts the Flask server
+    """
+    # Flask endpoint for user interaction
+    logger.debug("Entering run_flask()")
+    # app.run(port = str(str(fport)), debug=False)
+    app.run(port = str(str(fport)), debug=True)
+    # app.run(port=5000, debug=True, use_reloader=False)
+    logger.debug("Exiting run_flask()")
+
+
+if __name__ == '__main__':
+    # Run the Flask application
+    run_flask()
+
+

smartassist/src/enums.py

@@ -0,0 +1,18 @@
+# Some useful constants to be used in the code
+
+from enum import Enum
+
+class LogLevel(Enum):
+    DEBUG = 'DEBUG'
+    INFO = 'INFO'
+    WARNING = 'WARNING'
+    ERROR = 'ERROR'
+    CRITICAL = 'CRITICAL'
+
+LOG_LEVEL_MAPPING = {
+    LogLevel.DEBUG: 10,
+    LogLevel.INFO: 20,
+    LogLevel.WARNING: 30,
+    LogLevel.ERROR: 40,
+    LogLevel.CRITICAL: 50
+}

smartassist/src/html/client.html

@@ -0,0 +1,72 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Ollama Chat</title>
+    <link rel="stylesheet" href="/css/clientstyle.css"> 
+    <!-- <link rel="stylesheet" href="python_test/smartassist/src/css/clientstyle.css">  -->
+</head>
+
+<body>
+    <h1>Ollama Chat</h1>
+
+    <div class="dropdown">
+        <button class="dropbtn" id="selected-endpoint">Select Endpoint/LLM</button>
+        <div class="dropdown-content" id="endpoint-dropdown"></div>
+    </div>
+
+    <div id="chatbox">
+        <!-- messages will be rendered here -->
+    </div>
+    <textarea id="userInput" placeholder="Type your message..." rows="5"></textarea>
+    <button id="sendButton" onclick="window.frontendApi.sendMessage()">Send</button>
+
+
+    <!-- Marked-it for markdown rendering -->
+    <script src="https://cdn.jsdelivr.net/npm/markdown-it@14.1.0/dist/markdown-it.min.js"></script>
+    <script src="https://cdn.jsdelivr.net/npm/markdown-it@14/dist/markdown-it.min.js"></script>
+
+
+
+    <!-- Include MathJax library to render mathematical notation -->
+    <script id="MathJax-script" async src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
+    <script>
+        window.MathJax = {
+          loader: { load: ['input/tex', 'output/chtml'] },
+          tex: {
+            packages: ['base', 'ams'],
+            inlineMath: [['$', '$']]
+          }
+        };
+    </script>
+
+    <script>
+        const chatContainer = document.getElementById('chatbox');
+
+        // Handle resize events
+        window.addEventListener('resize', 
+            function() {
+                chatContainer.style.height = 'auto'; 
+        });
+
+        const userInputElement = document.getElementById('userInput');
+
+        userInputElement.addEventListener('keydown', function(event) {
+            if (event.shiftKey && event.key === 'Enter') { // Shift+Enter for newline
+                event.preventDefault(); 
+                userInputElement.value += '\n';
+            } else if (event.key === 'Enter') {  // Enter to send message
+                window.frontendApi.sendMessage();
+                userInputElement.value = ''; // Clear the input field after sending
+                event.preventDefault();
+            }
+        });
+
+    </script>
+
+    <!-- Get the javascript handling communication with the backend -->
+    <script src="/js/frontend.js"></script>
+</body>
+</html>
+

smartassist/src/startservices.py

@@ -0,0 +1,118 @@
+# Start all services
+import subprocess
+import os
+import yaml
+import json
+import socket
+import urllib.parse
+from backend import run_flask
+import logging
+import requests
+import utils
+from utils import GlobalState
+from enums import LogLevel
+
+global_state = GlobalState() # Configure root logger. The level will be adjusted later based on config file
+logger = global_state.get_logger(__name__)  # Logger for this module, inherit properties of the root logger
+
+def configure():
+    """
+    Reads YAML configruation file into dictionary, parse it and fill all referenceed
+    environment variables with their values.
+    """
+    ####################################
+    # Read YAML config
+    ####################################
+    # Load configuration file that defines parameters for services
+    with open('./smartassist/config/smartassist.yaml') as f:
+        config = yaml.safe_load(f)
+    
+    def resolve_env_var(value):
+        if isinstance(value, str) and value.startswith("${") and value.endswith("}"):
+            env_var_name = value[2:-1]  # Extract name between ${}
+            return os.getenv(env_var_name, None)
+        return value
+    
+    def update_value(value):
+        if isinstance(value, dict): # Dictionaries need recursive check
+            return update_dict_with_env_vars(value)
+        elif isinstance(value, list): # Lists must be traversed element by element
+            return [update_value(item) for item in value]
+        elif isinstance(value, str): # If value is a string it might be an environmnet variable
+            return resolve_env_var(value)
+        else: # Anything else, just keep the old value
+            return value
+
+    def update_dict_with_env_vars(d): # Check all keys in d
+        for key in d: # Iterate over all keys in the dictionary. The keys seen are all at the top-level of d
+            # logger.info(f"key investigated now: {key}")
+            d[key] = update_value(d[key])
+        return d
+    
+    updated_config = update_dict_with_env_vars(config)
+ 
+    ####################################
+    # Extract global logging level
+    ####################################
+    if isinstance(updated_config.get('logging'), dict): # Look for 'logging' key in config file
+        logging_config = updated_config['logging'] 
+        if isinstance(logging_config.get('level'), str): # Set to value of the yaml file if specified
+            # global_state.set_log_level(logging_config['level'])
+            global_state.set_log_level(LogLevel(logging_config['level']))
+            logger.info("configure(): This logger now has effective log level %s", logger.getEffectiveLevel())
+
+    ####################################
+    # Extract models (server url, api_key, model, et cetera)
+    ####################################
+    if isinstance(updated_config.get('backend'),dict): # Extract backend info from dictionary
+        global_state.set_backend(backend=updated_config.get('backend'))
+        logger.debug("backend = \n{}".format(json.dumps(global_state.get_backend(), indent=4)))
+        logger.debug(f"Backend API endpoint is set to: {global_state.get_backend_api_ep()}")
+    
+    preferred_ep = updated_config.get('preferred_ep', None) # Get the preferred endpoint if specified, otherwise None
+
+    if isinstance(updated_config.get('endpoints'), list):  # Extract info on endpoint, model, url, provider et cetera from list
+        global_state.set_endpoints(endpoints=updated_config.get('endpoints'))  # Extract and set list of endpoints
+        # logger.debug("endpoints = \n{}".format(json.dumps(global_state.get_endpoints(), indent=4)))
+        global_state.fetch_models()
+        endpoints = global_state.get_endpoints()
+        for endpoint in endpoints: # Set default LLM for each endpoint
+            available_llms = global_state.get_list_of_available_llms(endpoint=endpoint)
+            llm = next(iter(available_llms),None) # First available LLM or None. Default for AUTODETECT and requests for non-existing LLMs 
+            logger.debug(f"url {endpoint['url']} = {available_llms}")
+            if endpoint["model"] in available_llms: # Check if specific LLM requested, AUTODETECT evaluates to False
+                llm = endpoint["model"] 
+            endpoint["default_llm"] = llm
+ 
+        if preferred_ep:   # If preferred_ep is specified, set it as the default endpoint
+            list_of_eps = global_state.get_endpoints_with_key_value("title", preferred_ep) # Should only be one element in the list...
+            default_endpoint = next(iter(list_of_eps),None) # Same as default_endpoint = list_of_eps[0] if list_of_eps else None
+        else:
+            default_endpoint = next(iter(endpoints),None) # Set default_endpoint to first endpoint from list of all endpoints
+        default_llm = default_endpoint["default_llm"] # Get default LLM for default_endpoint
+        default_ulr = default_endpoint["url"] # Get ulr of default_endpoint
+        global_state.set_host_url(default_ulr) # Set initial host to the first item in endpoints (or None)
+        global_state.set_llm(default_llm) # Set which llm to use
+        logger.debug(f"Desired default endpoint: {default_ulr},\tDesired default LLM: {default_llm}")
+        logger.debug(f"Returned default endpoint: {global_state.get_host_url()},\tReturned default LLM: {global_state.get_llm()}")
+
+    return updated_config            
+
+
+def start_backend(config):
+    parsed_url = urllib.parse.urlparse(config['backend']['url'])
+    # hostname = parsed_url.netloc.split(':')[0]  # Split by ':' and take the first part, i.e., 'localhost', IP, or domain name
+    port = parsed_url.port  # This is the server port
+    logger.debug('Backend parsed url set to {}'.format(parsed_url))
+    logger.debug('Backend port set to {}'.format(port))
+
+    try:
+        run_flask(fport = port)
+    except Exception as e:
+        logger.error("Failed to start backend: %s", str(e)) # Corresponds to print(f"Failed to start backend: {e}")
+
+if __name__ == '__main__':
+    conf = configure() # Read config from file and set up config dict
+    # logger.debug('conf dictionary set to \n{}'.format(json.dumps(conf, indent=4)))
+    # start_frontend(config=conf) # Not needed as we are using Flask for backend now
+    start_backend(config=conf)

smartassist/src/static/css/clientstyle.css

@@ -0,0 +1,118 @@
+body {
+    font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
+    background-color: #f4f4f4;
+    display: flex;
+    flex-direction: column;
+    align-items: center;
+    min-height: 100vh;
+    margin: 0;
+}
+
+h1 {
+    color: #333;
+    margin-bottom: 20px;
+}
+
+#chatbox {
+    width: calc(50% - 60px); /* Adjust width for input and button */
+    /* max-width: 500px; */
+    height: 600px;
+    /* background-color: #fff8bc; */
+    background-color: #ffffff;
+    border-radius: 10px;
+    padding: 20px;
+    box-shadow: 0 4px 8px rgba(0,0,0,0.1);
+    overflow: auto; /* Allow horizontal and vertical scrolling of the chatbox */
+    resize: both; /* Allow resizing vertically */
+    border: 1px solid #ccc; /* Add a thin grey border around chatbox */
+    margin-bottom: 20px; /* Add some space between chatbox and userInput */
+    font-size: 14px; /* Decrease font size to 14 pixels */
+}
+
+.message {
+    margin-bottom: 15px;
+}
+
+.user-message {
+    background-color: #9cc1ecbb;
+    padding: 10px 15px;
+    border-radius: 10px;
+    text-align: left; /* Align user messages to the left */
+}
+
+.ai-response {
+    /* background-color: #f0f8ff; */
+    background-color: #f5ecd0;
+    padding: 10px 15px;
+    border-radius: 10px;
+    text-align: left; /* Align AI responses to the left */
+}
+
+#userInput {
+    width: calc(50% - 60px); /* Adjust width for input and button */
+    padding: 10px;
+    border: 1px solid #ccc;
+    border-radius: 5px;
+    margin-bottom: 10px;
+    font-family: 'Courier New', Courier, monospace; /* Fixed width typeface */
+}
+
+#userInput:focus {
+    outline: none;
+    border-color: #66afe9; /* Blue outline on focus */
+}
+
+button[onclick="window.frontendApi.sendMessage()"] {
+    background-color: #4CAF50; /* Green */
+    border: none;
+    color: white;
+    padding: 10px 20px;
+    text-align: center;
+    text-decoration: none;
+    display: inline-block;
+    font-size: 16px;
+    border-radius: 5px;
+    cursor: pointer;
+    transition: background-color 0.3s; /* Smooth transition effect */
+}
+
+button[onclick="window.frontendApi.sendMessage()"]:hover {
+    background-color: #b2b2b2; /* Light Grey on hover */
+}
+
+button[onclick="window.frontendApi.sendMessage()"]:active {
+    background-color: #6f6f6f; /* Dark Grey when clicked */
+}
+
+
+.dropdown {
+    position: relative;
+    display: inline-block;
+  }
+  
+  .dropdown-content {
+    display: none;
+    position: absolute;
+    background-color: #f9f9f9;
+    /* min-width: 160px; */
+    box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
+    z-index: 1;
+    width: auto; /* Add this property */
+  }
+  
+  .dropdown-content a {
+    color: black;
+    /* padding: 12px 16px; */
+    padding: 6px 8px;
+    text-decoration: none;
+    display: block;
+    font-size: 0.7rem; /* Decrease font size relative to root element */
+    line-height: 0.5; /* Decrease line height to reduce spacing */
+    white-space: nowrap; /* Add this property */
+  }
+  
+  .dropdown-content a:hover {background-color: #f1f1f1;}
+  
+  .dropdown:hover .dropdown-content {
+    display: block;
+  }

smartassist/src/static/js/frontend.js

@@ -0,0 +1,150 @@
+// Get the user input element from the DOM
+const chatbox = document.getElementById('chatbox');
+const userInput = document.getElementById('userInput');
+const parser = window.markdownit({
+  linkify: true,
+  strikethrough: true,
+});
+parser.enable(['table']);
+
+let apiEndpoint; // Make variable available outside of the scope of the event listener
+let useModel; // Make variable available outside of the scope of the event listener
+
+const frontendApi = {
+  // Define a function to send the user's message to the AI
+  sendMessage: function() {
+    if (!window.apiEndpoint || !window.useModel) { // Check if we're ready before proceeding
+      console.error("Not ready yet. Please wait for apiEndpoint and useModel to be set.");
+      return;
+    }
+    
+    // Get the user's input message and trim any whitespace
+    const query = userInput.value.trim();
+    // Check if the message is not empty
+    if (query !== '') {
+      fetch(window.apiEndpoint, {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        // body: JSON.stringify({ query }), // Add these parameters here
+        body: JSON.stringify({ query, model: window.useModel }), // Add these parameters here
+      })
+      .then(response => response.json())
+      .then(data => {
+        // Get the AI's response from the API data
+        const aiResponse = data.response;
+        // Render the user's original message in the chatbox
+        this.renderMessage(query, 'user-message');
+        // Render the AI's response in the chatbox
+        this.renderMessage(aiResponse, 'ai-response');
+        // Clear the user input field for the next message
+        userInput.value = '';
+      })
+      .catch(error => console.error('Error sending message:', error));
+    }
+  },
+  // Define a function to render a message in the chatbox with a specific class name
+  renderMessage: function(text, className) {
+    // Create a new div element to hold the message
+    const messageElement = document.createElement('div');
+    // Add the specified class name to the element
+    messageElement.className = className;
+    // Use the markdown-it parser
+    const html = parser.render(text);
+    messageElement.innerHTML = html;
+    // Append the message element to the chatbox immediately
+    chatbox.appendChild(messageElement);
+  },
+  // Make an AJAX request to fetch endpoint data from Flask backend
+  fillMenu: function() {
+    fetch('/api/endpoints')
+    .then(response => response.json())
+    .then(data => {
+      const dropdownContainer = document.getElementById('endpoint-dropdown');
+      
+      // Clear existing content
+      dropdownContainer.innerHTML = '';
+      
+      // Populate the dropdown menu with received data
+      data.forEach(endpoint => {
+        const linkElement = document.createElement('a');
+        linkElement.href = '';  // If attribute is set to '#', browser scrolls to top of page and reload
+        linkElement.onclick = () => frontendApi.setEndpointAndLlm(endpoint.title, endpoint.llm);
+        linkElement.textContent = `${endpoint.title} - ${endpoint.llm}`;
+        
+        dropdownContainer.appendChild(linkElement);
+      });
+    })
+    .catch(error => console.error('Error fetching endpoints:', error));
+  },
+  // Set the endpoint (remember, endpoint here is the 'title' of endpoin) and LLM variables
+  setEndpointAndLlm: function(title, llm) {
+    window.endpointTitle = title;
+    window.useModel = llm;
+    // Lets tell Flask about the new setting    
+    fetch('/api/select_endpoint_llm', {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({ title, llm }),
+      // body: JSON.stringify(`${{ title, llm }}`),
+    })
+    .then(response => response.json())
+    .then(data => {
+      // If everything went well, let's tell frontend about it
+      message = data.message;
+      console.log(message)
+      console.log(`Selected endpoint title: ${title}, LLM: ${llm}`);
+    })
+    .catch(error => console.error('Error setting endpoint and LLM:', error));
+  },
+}
+
+// Wait for the event listener to set apiEndpoint and useModel
+window.addEventListener('message', function(event) {
+  if (event.origin === 'http://localhost:5004') { // Make sure this matches your origin
+    const { apiEndpoint, useModel } = event.data;
+    console.log("fronend.js - API Endpoint: ", apiEndpoint);
+    console.log("fronend.js - use model: ", useModel);
+    window.apiEndpoint = apiEndpoint;
+    window.useModel = useModel;
+  }
+});
+
+// Wait for the DOM to be fully loaded before making the API available
+document.addEventListener('DOMContentLoaded', function() {
+  window.frontendApi = frontendApi;
+});
+
+
+// Make the button toggle colour when user presses Enter on keyboard
+const sendButton = document.getElementById('sendButton');
+document.addEventListener('keydown', function(event) {
+  if (event.key === 'Enter') {
+    sendButton.style.backgroundColor = '#6f6f6f'; // Dark Grey when Enter is pressed
+  }
+});
+document.addEventListener('keyup', function() {
+  sendButton.style.backgroundColor = ''; // Restore the original style when any key is released
+});
+
+// Get the dropdown button and the dropdown content elements
+const dropbtn = document.getElementById('selected-endpoint');
+const dropdownContent = document.getElementById('endpoint-dropdown');
+
+// Add event listeners to each dropdown item
+dropdownContent.addEventListener('click', (e) => {
+    if (e.target.tagName === 'A') { // Only respond to clicks on anchor tags
+        e.preventDefault(); // Prevent default link behavior, i.e., do NOT navigate to the link's URL when clicked
+        const selectedEndpoint = e.target.textContent;
+        dropbtn.textContent = selectedEndpoint; // Update the button's text
+        // You can also add code here to update the current endpoint in your application
+    }
+});
+
+function init() {
+  // Other initialization code here...
+  frontendApi.fillMenu();
+}
+
+document.addEventListener('DOMContentLoaded', init);
+
+

smartassist/src/templates/index.html

@@ -0,0 +1,54 @@
+<!DOCTYPE html>
+<html>
+<head>
+<!-- <title>Frontend</title> -->
+</head>
+<body>
+
+
+    <!-- This iframe will hold the content from client.html -->
+    <!-- Passing the API endpoint as a query parameter to the srcdoc attribute -->
+    <!-- srcdoc="{{ client_content }}?apiEndpoint={{ api_endpoint }}/"> -->
+
+    <iframe id="client-frame" 
+            style="width: 100%; height: 100vh;" 
+            srcdoc="{{ client_content }}">
+    </iframe>
+
+    <!-- <script>
+        // Extract apiEndpoint for use in your frontend code...
+        const apiEndpoint = '{{ api_endpoint }}';  // Templating syntax (Jinja2)
+        const useModel = '{{ use_model }}';  // Templating syntax (Jinja2)
+        // Tell the iframe about the apiEndpoint
+        document.getElementById('client-frame').contentWindow.apiEndpoint = apiEndpoint;
+        document.getElementById('client-frame').contentWindow.useModel = useModel;
+        console.log("index.html - API Endpoint: ", apiEndpoint);
+        console.log("index.html - use model: ", useModel);
+    </script> -->
+
+    <script>
+        // Extract apiEndpoint for use in frontend.js
+        const apiEndpoint = '{{ api_endpoint }}';  // Templating syntax (Jinja2)
+        const useModel = '{{ use_model }}';  // Templating syntax (Jinja2)
+        window.addEventListener('load', function() {
+            const clientFrame = document.getElementById('client-frame').contentWindow;
+            clientFrame.postMessage({ apiEndpoint, useModel }, '*'); // Send the data to the iframe
+        });
+        console.log("index.html - API Endpoint: ", apiEndpoint);
+        console.log("index.html - use model: ", useModel);
+    </script>
+
+    <!-- Responsive scaling and some padding -->
+    <script>
+        const clientFrame = document.getElementById('client-frame');
+      
+        function resizeIframe() {
+            clientFrame.style.height = window.innerHeight - 50 + 'px'; // Adjust the subtraction for padding/margins if needed
+        }
+      
+        window.addEventListener('resize', resizeIframe);
+        resizeIframe(); // Call it once on page load
+    </script>
+
+</body>
+</html>

smartassist/src/utils.py

@@ -0,0 +1,298 @@
+# This module contains definitions of variables, functions, classes, et cetera, that are
+# imported to more than one other module. The rational for defining these things here
+# is that it is easier to avoid circular imports when they are defined in a central location.
+import logging
+import json
+import requests
+from typing import Optional
+from enums import LogLevel, LOG_LEVEL_MAPPING
+
+class GlobalState:
+    """
+     This class holds various variables and methods which are accessible across 
+     different modules in the Python project using the Singleton design pattern. 
+     This ensures that only one instance of the class is created and shared among 
+     all modules, preventing circular imports and providing a centralized location 
+     for managing shared resources.
+    """
+    _instance = None  # Private class attribute to hold the single instance of the class
+
+    def __new__(cls) -> 'GlobalState':
+        """
+        Create a new instance of the GlobalState class.
+        
+        This is a singleton implementation, so only one instance will be created.
+        """
+        if cls._instance is None:
+            cls._instance = super(GlobalState, cls).__new__(cls)
+            cls._instance.log_level = 'INFO'  # Default logging level
+            cls._instance.logger = logging.getLogger() # Get root logger for the caller module
+            handler = logging.StreamHandler() # Or other handler (FileHandler for logs to file)
+            formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
+            handler.setFormatter(formatter)
+            cls._instance.logger.addHandler(handler)
+            cls._instance.logger.setLevel(getattr(logging, cls._instance.log_level))  # Initialize root logger level
+            cls._instance.logger.info(" __new__(cls): Logger in GlobalState created: %s", cls._instance.logger)
+            cls._instance.host_url = None # Currently used LLM host
+            cls._instance.llm = "phi3:mini" # Default LLM for queries. TODO: Check with ollama server that it actually exists
+            # cls._instance.backend_api_ep = "http://localhost:5005/api/chat" # Default backend API endpoint
+            # Try making things more aligned with the outline of the yaml file
+            cls._instance.backend = dict() # A dictionary that holds info on which server the clients connect to
+            cls._instance.endpoints = [] # A list that holds info on which endpoints are available for use (server url, model name, provider et cetera)
+            # logging - already done in __new__, perhaps change layout later
+
+        return cls._instance
+
+
+    def configure_logging(self, level: Optional[LogLevel] = None) -> None:
+        """
+        Configure the logging system for this project.
+        
+        Args:
+            level (LogLevel): The log level to use. If None, uses the default log level set in `self.log_level`.
+        
+        Notes:
+            This method sets up logging for the project and logs a message at the debug level indicating the effective log level.
+        """
+        if level == None:
+            level = self.log_level
+        if isinstance(level, LogLevel):
+            logging.info(f"Trying to set up logging with level {level}")
+            numeric_level = LOG_LEVEL_MAPPING[level]
+            if numeric_level is None:
+                raise ValueError("Invalid log level")
+        self.logger.setLevel(numeric_level)
+        self.logger.debug(f"utils.py -- configure_logging(): effective log level is {level} which is {self.logger.getEffectiveLevel()}")
+
+
+    def set_log_level(self, level: LogLevel) -> None:
+        """
+        Set the log level for this project.
+
+        Args:
+            level (LogLevel): The new log level to use. Can be one of the evels defined in enum.py (e.g., DEBUG, INFO, WARNING, CRITICAL etc.).
+
+        Notes:
+            This method updates the `self.log_level` attribute and calls `configure_logging()` to apply the change.
+        """    
+        self.log_level = level
+        self.configure_logging()
+
+    def get_log_level(self) -> LogLevel:
+        """
+        Get the current log level.
+
+        Returns:
+            str: The current log level (e.g., 'DEBUG', 'INFO', 'WARNING', etc.).
+        """
+        return self.log_level
+    
+    def get_effective_log_level(self) -> int:
+        """
+        Get the effective log level of the logger.
+
+        Returns:
+            int: The numeric value of the effective log level.
+        """
+        return self.logger.getEffectiveLevel()
+    
+    def get_logger(self, module_name: Optional[str] = None) -> logging.Logger:
+
+        """
+        Get a logger instance based on the module name.
+
+        Args:
+            module_name (str): The name of the module to get a logger for. If None, uses the current module name (`__name__`).
+
+        Returns:
+            Logger: A logger instance configured for the specified module.
+        """
+        if module_name is None:
+            module_name = __name__
+        logger = logging.getLogger(module_name)
+        return logger
+    
+    def set_host_url(self, url: str = "http://localhost:11434") -> None:
+        """
+        Set the URL of the host to which LLM requests are sent.
+
+        Args:
+            url (str): The new URL to use. Defaults to 'http://localhost:11434' if not specified.
+        """
+        self.host_url = url
+
+    def get_host_url(self) -> str:
+        """
+        Get the URL of the host currently used for LLMs.
+
+        Returns:
+            str: The URL of the current host.
+        """
+        return self.host_url
+    
+    def get_host_title(self) -> str:
+        """
+        Get the title of the host currently used for LLMs.
+        There must be a 1-to-1 mapping from host_url to host_title.
+
+        Returns:
+            str: The title of the current host.
+        """
+        endpoints = self.get_endpoints_with_key_value('url', self.get_host_url())
+        if len(endpoints) != 1:
+            raise ValueError(f"Expected exactly one endpoint with url '{self.get_host_url()}', found {len(endpoints)}")
+        return endpoints[0]["title"]
+    
+    def set_llm(self, model_name: str = "phi3:mini") -> None:
+        """
+        Set the LLM to use for queries.
+
+        Args:
+            model_name (str): The name of the LLM to use. Defaults to 'phi3:mini' if not specified.
+        """
+        self.llm = model_name
+
+    def get_llm(self) -> str:
+        """
+        Get the current LLM used for queries.
+
+        Returns:
+            str: The name of the current LLM.
+        """
+        return self.llm
+    
+    def set_backend(self, backend: Optional[dict] = None) -> None:
+
+        """
+        Set the backend server that web clients connect to.
+
+        Args:
+            backend (dict): A dictionary containing information about the backend server. If None, resets the backend server to its default value.
+        """
+        self.backend = backend
+
+    def get_backend(self) -> dict:
+        """
+        Get the current backend server used by web clients.
+
+        Returns:
+            dict: A dictionary containing information about the current backend server.
+        """
+        return self.backend
+    
+    def get_backend_api_ep(self) -> str:
+        """
+        Get the API endpoint of the backend server.
+
+        Returns:
+            str: The URL of the API endpoint.
+        """
+        return self.backend["url"]+self.backend["api"]
+
+    def set_endpoints(self, endpoints: Optional[list[dict]] = None) -> None:
+        """
+        Set the list of endpoints used by this object.
+
+        Args:
+            endpoints (list): A list of endpoint dictionaries. Each dictionary should contain information about an endpoint.
+                If None, resets the endpoints to their default value.
+
+        Raises:
+            ValueError: If endpoints is not a list.
+
+        Notes:
+            Endpoints can be reset to their default value by passing None as the argument.
+        """
+        if endpoints is not None:
+            if not isinstance(endpoints, list):
+                raise ValueError("Endpoints must be a list, even if there is just one model")
+        self.endpoints = endpoints
+   
+    def get_endpoints(self) -> list[dict]:
+        """
+        Get the complete list of endpoints.
+        
+        Returns:
+            List of endpoints
+        """
+        return self.endpoints
+    
+    def get_endpoints_with_key(self, key: str) -> list[dict]:
+        """
+        Returns a list of endpoint dictionaries that contain the specified key.
+
+        Args:
+            key (str): The key to search for in the endpoint dictionaries.
+
+        Returns:
+            List[Dict]: A list of endpoint dictionaries containing the specified key.
+        """
+        return [ep for ep in self.endpoints if key in ep]
+
+    def get_endpoints_with_key_value(self, key: str, value: any) -> list[dict]:
+        """
+        Returns a list of endpoint dictionaries that contain the specified key-value pair.
+
+        Args:
+            key (str): The key to search for in the endpoint dictionaries.
+            value (Any): The value to search for in the endpoint dictionaries.
+
+        Returns:
+            list[dict]: A list of endpoint dictionaries containing the specified key.
+        """
+        return [ep for ep in self.endpoints if key in ep and value == ep[key]]
+    
+    def fetch_models(self) -> None:
+        """
+        Fetch models from endpoints and update the endpoint dictionaries.
+        Returns:
+            None
+        """
+        logger = logging.getLogger(__name__)
+
+        for endpoint in self.endpoints:
+            try:
+                if endpoint["provider"] == "ollama":
+                    headers = {
+                        "Content-Type": "application/json",
+                    }
+                    if "requestOptions" in endpoint:  # Check if authentication is needed
+                        headers.update({
+                            "Authorization": endpoint["requestOptions"]["headers"]["Authorization"]
+                        })
+
+                    models_response = requests.get(endpoint["url"] + "/api/tags", headers=headers)
+                    models_response.raise_for_status()  # Raise an exception for HTTP errors
+
+                    try:
+                        models = models_response.json()
+                    except json.JSONDecodeError as e:
+                        logger.error(f"Failed to parse JSON response: {e}")
+                        continue
+
+                    if isinstance(models, dict) and 'error' in models: # Unclear if requests to any API actually add this in the response
+                        logger.error('Error fetching models from backend: %s', models['error'])
+                    else:
+                        endpoint["models"] = models.get("models", [])  # Get the list of models directly
+            except requests.exceptions.RequestException as e:
+                logger.error(f"Request error: {e}")
+            except Exception as e:
+                logger.error(f"Unexpected error: {e}")
+
+        return # No value returned
+
+    def get_list_of_available_llms(self, endpoint: Optional[dict] = None) -> Optional[list[str]]:
+        """
+        Returns a sorted list of Large Language Models (LLMs) available at the specified endpoint.
+
+        Args:
+            endpoint (dict): Optional endpoint dictionary to retrieve LLMs from. If not provided, will use internal endpoint configuration.
+
+        Returns:
+            list: A sorted list of LLM names (strings). Returns None if no LLMs are found or endpoint is invalid.
+        """        
+        llm_list = None
+        if isinstance(endpoint["models"], list):
+            llm_list = sorted([list_item['name'] for list_item in endpoint["models"]], key=str.lower)
+        return llm_list
+    

web/download_images_gpt4o.py

@@ -0,0 +1,76 @@
+import os
+import requests
+from bs4 import BeautifulSoup
+from urllib.parse import urljoin, urlparse
+import base64
+import re
+
+def download_image(url, folder_path):
+    if not os.path.isdir(folder_path):
+        os.makedirs(folder_path)
+    
+    try:
+        response = requests.get(url, stream=True)
+        response.raise_for_status()  # Kontrollera om förfrågan lyckades
+    except requests.RequestException as e:
+        print(f"Failed to retrieve image {url}: {e}")
+        return
+
+    filename = os.path.join(folder_path, os.path.basename(urlparse(url).path))
+    with open(filename, 'wb') as file:
+        for chunk in response.iter_content(1024):
+            file.write(chunk)
+    print(f"Downloaded: {filename}")
+
+def save_base64_image(data_url, folder_path, count):
+    if not os.path.isdir(folder_path):
+        os.makedirs(folder_path)
+    
+    match = re.match(r'data:image/(?P<ext>[^;]+);base64,(?P<data>.+)', data_url)
+    if match:
+        ext = match.group('ext')
+        data = match.group('data')
+        img_data = base64.b64decode(data)
+        filename = os.path.join(folder_path, f'image_{count}.{ext}')
+        with open(filename, 'wb') as file:
+            file.write(img_data)
+        print(f"Downloaded: {filename}")
+    else:
+        print(f"Invalid base64 image data: {data_url}")
+
+def download_all_images(html_content, base_url, folder_path):
+    soup = BeautifulSoup(html_content, 'html.parser')
+    img_tags = soup.find_all('img')
+    
+    count = 0
+    for img in img_tags:
+        img_url = img.get('src')
+        if not img_url:
+            continue
+
+        if img_url.startswith(('http://', 'https://')):
+            img_url = urljoin(base_url, img_url)
+            print(f"Attempting to download image: {img_url}")
+            download_image(img_url, folder_path)
+        elif img_url.startswith('data:image/'):
+            print(f"Attempting to save base64 image: {img_url[:30]}...")  # Print only the start of the data URL
+            count += 1
+            save_base64_image(img_url, folder_path, count)
+        else:
+            print(f"Ignoring non-http URL: {img_url}")
+
+def main():
+    url = input("Enter the URL of the webpage: ")
+    folder_path = os.path.expanduser("~/Downloads/downloaded_images")
+    
+    try:
+        response = requests.get(url)
+        response.raise_for_status()  # Kontrollera om förfrågan lyckades
+    except requests.RequestException as e:
+        print(f"Failed to retrieve webpage {url}: {e}")
+        return
+
+    download_all_images(response.content, url, folder_path)
+
+if __name__ == "__main__":
+    main()

web/ladda_hem_bilder.py

@@ -0,0 +1,62 @@
+import os
+import re
+from bs4 import BeautifulSoup
+import requests
+from urllib.parse import urljoin
+import base64
+
+def ladda_ner_bilder(url):
+    # Hämta HTML-sidan
+    svar = requests.get(url)
+    soup = BeautifulSoup(svar.text, 'html.parser')
+
+    # Hitta alla bilder
+    bilder = []
+    for img in soup.find_all('img'):
+        src = img.get('src')
+        if src:
+            bilder.append(src)
+
+    # Hantera inline-bilder i base64
+    INLINE_BILD_MÖNSTER = r'data:image/(.*?);base64,(.*)'
+    matcher = re.compile(INLINE_BILD_MÖNSTER)
+    for match in matcher.finditer(svar.text):
+        bild_typ = match.group(1)
+        bild_data = match.group(2)
+        bilder.append(f"data:{bild_typ};base64,{bild_data}")
+
+    # Ladda ner bilderna
+    bild_katalog = os.path.expanduser("~/Downloads/bilder")
+    if not os.path.exists(bild_katalog):
+        os.makedirs(bild_katalog)
+
+    for bild_url in bilder:
+        if not bild_url.startswith('http'):
+            bild_url = urljoin(url, bild_url)
+
+        if bild_url.startswith('data:'):
+            # Dekodera base64-strängen och spara den som en bild
+            format, data = bild_url.split(';base64,')
+            data = base64.b64decode(data)
+            filnamn = 'inline_' + str(len(bilder)) + '.gif'
+            with open(os.path.join(bild_katalog, filnamn), 'wb') as f:
+                f.write(data)
+        else:
+            svar = requests.get(bild_url)
+            if svar.status_code == 200:
+                filnamn = os.path.basename(bild_url).split('?')[0]
+                with open(os.path.join(bild_katalog, filnamn), 'wb') as f:
+                    f.write(svar.content)
+            print(f"Bilden {filnamn} har laddats ner till {bild_katalog}.")
+
+def main():
+    url = input("Ange URL till sidan från vilken du vill hämta bilder: ")
+    if not url.startswith('http'):
+        url = 'http://' + url
+    try:
+        ladda_ner_bilder(url)
+    except Exception as e:
+        print(f"Fel inträffade: {e}")
+
+if __name__ == "__main__":
+    main()

web/requirements_web.txt

@@ -0,0 +1,2 @@
+BeautifulSoup4
+requests