Skip to main content

Shell Script: Append And Minify Your Code Into Files for AI to Analyze

Published Date
Author Douglas Karr

Shell Script to Combine Code Files for AI

If you’re a developer, you’re likely getting a lot of assistance from generative AI (GenAI). However, all AI platforms have maximum file size and limit the number of files you can upload. This can be frustrating if you’re working with a directory that may have dozens of small files that are organized well.

I have encountered this issue repeatedly when working on WordPress plugins or child theme modifications. I started to wonder if there was an easy way to combine all the code and organize it in text files that I could upload easily. And there was!

Shell Script to Combine Code Files

I developed this minify.sh shell script that can be run on Linux, Unix, or MacOS from a terminal command within your working folder. I’ve utilized this to upload code for review on Gemini, ChatGPT, and Claude with no issues.

#!/bin/bash

# Configuration
MAX_SIZE=$((650 * 1024))  # 650KB in bytes
FILE_EXTENSIONS=("php" "js" "html" "css")

# Get current folder name for output prefix
OUTPUT_PREFIX=$(basename "$(pwd)")

# Initialize variables for each file type
declare -A file_counters=( ["php"]=1 ["js"]=1 ["html"]=1 ["css"]=1 )
declare -A current_sizes=( ["php"]=0 ["js"]=0 ["html"]=0 ["css"]=0 )
declare -A output_files

# Function to minify content based on file type
minify_content() {
    local file=$1
    local extension="${file##*.}"
    local content

    case "$extension" in
        php)
            content=$(cat "$file" | \
                sed -e 's/\/\*.*\*\///g' \
                    -e 's/\/\/.*$//' \
                    -e '/^[[:space:]]*$/d' \
                    -e 's/[[:space:]]\+/ /g' \
                    -e 's/> </></g' \
                    -e 's/\r//g')
            ;;
        js)
            content=$(cat "$file" | \
                sed -e 's/\/\*.*\*\///g' \
                    -e 's/\/\/.*$//' \
                    -e '/^[[:space:]]*$/d' \
                    -e 's/[[:space:]]\+/ /g' \
                    -e 's/;[[:space:]]/;/g' \
                    -e 's/,[[:space:]]/,/g' \
                    -e 's/\r//g')
            ;;
        css)
            content=$(cat "$file" | \
                sed -e 's/\/\*.*\*\///g' \
                    -e '/^[[:space:]]*$/d' \
                    -e 's/[[:space:]]\+/ /g' \
                    -e 's/:[[:space:]]/:/g' \
                    -e 's/,[[:space:]]/,/g' \
                    -e 's/;[[:space:]]/;/g' \
                    -e 's/{[[:space:]]*/{/g' \
                    -e 's/}[[:space:]]*/}/g' \
                    -e 's/[[:space:]]*{/{/g' \
                    -e 's/[[:space:]]*}/}/g' \
                    -e 's/\r//g')
            ;;
        html)
            content=$(cat "$file" | \
                sed -e 's/<!--.*-->//g' \
                    -e '/^[[:space:]]*$/d' \
                    -e 's/[[:space:]]\+/ /g' \
                    -e 's/> </></g' \
                    -e 's/\r//g')
            ;;
    esac
    echo "$content"
}

# Function to write file header
write_file_header() {
    local full_path=$1
    local type=$2
    echo "/* File: $full_path */" >> "${output_files[$type]}"
}

# Function to process files of a specific type
process_file_type() {
    local type=$1
    local counter_ref="file_counters[$type]"
    local size_ref="current_sizes[$type]"
    
    # Initialize first file for this type
    output_files[$type]="${OUTPUT_PREFIX}-${type}-${file_counters[$type]}.txt"

    find . -type f -name "*.$type" -print0 | while IFS= read -r -d '' file; do
        # Minify and get content
        local minified_content=$(minify_content "$file")
        local file_size=${#minified_content}
        
        # Check if we need to start a new file
        if (( current_sizes[$type] + file_size > MAX_SIZE )) && (( current_sizes[$type] > 0 )); then
            echo "Size limit reached for $type (${current_sizes[$type]} bytes). Creating new file..."
            file_counters[$type]=$((file_counters[$type] + 1))
            output_files[$type]="${OUTPUT_PREFIX}-${type}-${file_counters[$type]}.txt"
            current_sizes[$type]=0
        fi
        
        # Process the file
        echo "Processing $type file: $file (Original size: $(wc -c < "$file") bytes, Minified: $file_size bytes)"
        
        write_file_header "$(realpath "$file")" "$type"
        echo "$minified_content" >> "${output_files[$type]}"
        echo -e "\n" >> "${output_files[$type]}"
        
        current_sizes[$type]=$((current_sizes[$type] + file_size))
        echo "Current $type file size: ${current_sizes[$type]} bytes"
    done
}

# Main execution
echo "Starting file minification and combination process..."
echo "Using folder name '${OUTPUT_PREFIX}' as prefix for output files..."

# Process each file type separately
for type in "${FILE_EXTENSIONS[@]}"; do
    echo "Processing $type files..."
    process_file_type "$type"
done

echo "Processing complete! Files have been minified and combined by type."
echo "Generated files:"
ls -lh ${OUTPUT_PREFIX}-*.txt

Configuration

The script starts by defining some key parameters:

  • MAX_SIZE: This variable sets the maximum size (650KB) for each output file. This ensures that the combined files don’t become too large to handle.
  • FILE_EXTENSIONS: This array lists the file types the script will process: PHP, JavaScript, HTML, and CSS.
  • OUTPUT_PREFIX: This cleverly uses the current folder’s name as a prefix for the output files, keeping things organized.

The Minification Engine

The heart of the script lies in the minify_content function. It takes a file as input and, based on its extension, applies a series of sed commands to strip out comments, unnecessary whitespace, and line breaks. This results in a compressed version of the original code.

File Handling and Combining

The script uses associative arrays (declare -A) to keep track of file counters, current file sizes, and output file names for each file type. The process_file_type function iterates through each file type and performs the following steps:

  1. Finds all files: It uses the find command to locate all files with the specified extension.
  2. Minifies content: It calls the minify_content function to compress the file.
  3. Checks file size: It ensures that adding the current file doesn’t exceed the MAX_SIZE. If it does, a new output file is created.
  4. Writes to file: It adds a header with the original file path for debugging purposes and appends the minified content to the output file.

Putting It All Together

The main execution section of the script iterates through the FILE_EXTENSIONS array, calling the process_file_type function for each one. I even have it parse the different types of code in different files (PHP, JS, CSS, HTML) to make it easier. Working on a PHP issue? Just upload the file for PHP and ignore the rest. Finally, it lists the generated files with their sizes.

Combining minification and file combining significantly reduces file sizes and the number of files to upload to your favorite generative AI platform.

How To Use This Shell Script

  1. Save the Script: Copy the code provided in the article, paste it into a text editor, and save the file with a .sh extension (e.g., minify.sh).
  2. Make it Executable: Open your terminal or command prompt, navigate to the directory where you saved the script using the cd command, and make the script executable by running: chmod +x minify.sh.
  3. Navigate to the directory: Use the cd command to navigate to the directory with your code. One easy way to do this is to type cd and drag the folder from a Finder window to your terminal.
cd /Users/douglaskarr/Documents/my-folder
  1. Run the Script: Execute the script by typing the path where minify.sh is located and hitting Enter.
/Users/douglaskarr/Documents/minify.sh
  1. The script will output the following:
    • Progress messages as it processes each file type.
    • Information about original and minified file sizes.
    • Notifications when new output files are created due to size limits.
    • A list of the generated files with their sizes.
  1. Locate the Output Files:
    • The combined and minified files will be in the same directory where you ran the script.
    • They will be named using the current folder name as a prefix, followed by the file type, and a counter (e.g., myfolder-js-1.txt, myfolder-css-2.txt).

Important Notes:

  • File Types: The script currently handles PHP, JavaScript, HTML, and CSS files. You can modify the FILE_EXTENSIONS array in the script to include other file types if needed.
  • Maximum File Size: The MAX_SIZE variable determines the maximum size of each output file. Adjust this value if you need larger or smaller combined files.
  • Dependencies: This script relies on the sed command, which is typically available on Unix-like systems. If you’re on Windows, ensure you have a suitable environment like WSL, Cygwin, or Git Bash.
  • Customization: You can further customize the minification rules within the minify_content function to match your specific requirements.
  • Backup: It’s always a good practice to back up your original files before running this script, just in case.

By following these simple steps, you can leverage the power of this Bash script to combine and minify your code for your AI platform to review!

©2024 DK New Media, LLC, All rights reserved | Disclosure

Originally Published on Martech Zone: Shell Script: Append And Minify Your Code Into Files for AI to Analyze