(Legacy) LMTokenCook for MacOS (.dmg)

LMTokenCook for macOS

Version 1.0.0 (Initial Release)

by Steven Seagondollar, DropShock Digital LLC

---

Release Date: (Insert Actual Release Date, e.g., November 10, 2023)

🚀 Download LMTokenCook v1.0.0 for macOS

• Download LMTokenCook.dmg.zip (macOS Universal)
  (This link should point directly to the LMTokenCook.dmg.zip asset included in this GitHub Release.)

  Important Download Note: The LMTokenCook application for macOS is provided as an Apple Disk Image (.dmg), which has been compressed into a .zip archive. This extra zipping step helps ensure smoother downloads from platforms like GitHub and can prevent potential security blocks by some web browsers or macOS Gatekeeper that occasionally occur with direct .dmg file downloads. You will need to unzip the downloaded LMTokenCook.dmg.zip file first to access and use the enclosed LMTokenCook.dmg file.

---

👋 Welcome to LMTokenCook for macOS!

This is the first official release of LMTokenCook for the macOS platform. LMTokenCook is a powerful utility designed for AI power-users, developers, researchers, and writers who need to prepare large amounts of text-based content for use with Large Language Models (LLMs) that have web-based interfaces. It helps you consolidate various file types and intelligently divide them into manageable "servings" based on token limits.

This macOS version provides the full functionality of LMTokenCook through its intuitive Graphical User Interface (GUI).

For a complete understanding of LMTokenCook's capabilities, all its features, detailed usage instructions (including the GUI walkthrough and how to set optimal serving sizes), advanced configuration options, troubleshooting advice, and more, please consult the main project README.md file. This release note focuses specifically on getting started with this macOS package.

---

🍎 macOS Installation & First Launch Instructions

Follow these steps to install and run LMTokenCook on your Mac:

1. Download & Unzip:

   • Click the download link above to get the LMTokenCook.dmg.zip file.
   • Once downloaded (usually to your Downloads folder), double-click the .zip file. Your Mac's Archive Utility should automatically extract its contents, resulting in a file named LMTokenCook.dmg.

2. Mount the Disk Image:

   • Double-click the extracted LMTokenCook.dmg file. This will "mount" the disk image on your system, and a new Finder window will typically open, showing the contents of this mounted image (usually the LMTokenCook.app icon).

3. Install the Application:

   • From the Finder window that opened (showing the contents of the mounted DMG), drag the LMTokenCook.app icon and drop it into your main /Applications folder. (Your /Applications folder is usually found in the Finder sidebar or by selecting "Go > Applications" from the Finder menu bar at the top of your screen). This action copies the application to your Mac.

4. Eject the Disk Image:

   • After successfully copying the application to your Applications folder, you can eject the mounted "LMTokenCook" disk image. To do this, find its icon (either on your Desktop or in the Finder sidebar), then Ctrl-click (or right-click) on it and select "Eject".

5. First Launch (Important macOS Gatekeeper Step):
   Since LMTokenCook is distributed outside the Mac App Store and this initial release may not be notarized by Apple, macOS Gatekeeper (a security feature) will likely require you to explicitly approve running it the first time:

   • Navigate to your /Applications folder in Finder.
   • Locate LMTokenCook.app.
   • Right-click (or hold Control and click) on LMTokenCook.app.
   • From the context menu that appears, select "Open".
   • A dialog box may appear warning that "macOS cannot verify the developer..." or that the app is from an "unidentified developer." This is expected. Click the "Open" button in this dialog.
   • Alternative (if the "Open" button is not available in the warning): Go to System Settings > Privacy & Security. Scroll down to the "Security" section. You should see a message stating that LMTokenCook.app was blocked. Click the "Open Anyway" button next to this message. You might be asked for your Mac administrator password.
   • This approval is typically only needed for the very first launch. Subsequent launches can usually be done by double-clicking the app icon.

6. Using LMTokenCook:
   LMTokenCook is now ready to use! Launch it from your Applications folder. For guidance on its features and how to use the interface effectively, please see the "Usage Guide" section in the main README.md.

   

   The LMTokenCook interface on macOS.

---

✨ macOS Version Highlights (v1.0.0)

This initial macOS release of LMTokenCook delivers the core suite of tools designed to help you prepare and segment large text-based datasets for LLMs:

• Native macOS Application: A self-contained .app bundle for easy installation and a native experience on macOS, built with CustomTkinter.
• Full GUI Functionality: Access all LMTokenCook features through its user-friendly graphical interface, including:
  • Selection of input files or folders (via browse or drag-and-drop).
  • Choice of output parent directory.
  • Setting the crucial "Serving Size (Max Tokens)".
  • Options for keeping the master file, adding line numbers, and skipping empty lines.
• Intelligent File Processing: Supports text extraction from various formats including .txt, .md, code files, .docx, .pdf (text layer), and .ipynb.
• Accurate Tokenization: Uses OpenAI's tiktoken (cl100k_base) for reliable token counting.
• Organized Output: Creates timestamped output folders with a detailed manifest.json and your processed text file(s) (masterfile.txt and/or serving_XXX_of_YYY.txt).
• macOS Native File Dialogs: Uses standard macOS open/save dialogs for a familiar user experience.

(For a complete list of all application features and their detailed descriptions, please consult the "Key Features" section in the main README.md.)

---

🛠️ System Requirements for macOS

To ensure LMTokenCook runs smoothly on your Mac, please meet the following minimum requirements:

• Operating System: macOS 11.0 (Big Sur) or later.
  • LMTokenCook is built as a Universal application, meaning it should run natively on both Apple Silicon (M1/M2/M3 series) Macs and Intel-based Macs that meet the OS version requirement.
• Processor: Any Intel or Apple Silicon processor supported by a compatible macOS version.
• RAM (Memory): A minimum of 4GB of RAM is required. 8GB or more is recommended for better performance, especially when processing very large files or numerous documents.
• Hard Disk Space: Approximately 150-250MB of free disk space is required for the application itself. Sufficient additional disk space will be needed for your input files and the text output generated by LMTokenCook.
• Display: A screen resolution of 1280x800 pixels or higher is recommended for optimal viewing of the application interface.

---

⚠️ Known Considerations for This macOS Release (v1.0.0)

• Gatekeeper Warning on First Launch: As this application is distributed outside the Mac App Store and this initial release may not be notarized by Apple, you will encounter a security warning from Gatekeeper when you first try to open it. Please follow the "First Launch" instructions provided above (using Right-click > Open, or System Settings > Privacy & Security) to allow the application to run.
• Initial Verification Time: The very first time you launch LMTokenCook after installation, macOS may take a few extra moments to verify the application before it opens. Subsequent launches should be faster.
• PDF Text Extraction Limitations: Text extraction from PDF files relies on a proper text layer. LMTokenCook does not perform OCR on image-only PDFs. The quality of extraction can also vary with PDF complexity.
• Drag-and-Drop: While supported, if you encounter any issues with drag-and-drop, please use the "Browse..." buttons as the primary method for selecting files/folders.

---

🛟 Support & Feedback for macOS Version

If you encounter issues specific to this macOS version of LMTokenCook, or have feedback related to its performance or behavior on macOS:

• Primary Support: Please refer to the contact methods in the main README.md (Email: support@dropshockdigital.com, GitHub Issues for the project).
• When reporting macOS-specific issues, please include your macOS version (e.g., Sonoma 14.2.1), the LMTokenCook version (v1.0.0 for this release), and details about your Ma...

(Legacy) LMTokenCook for Windows (.exe)

LMTokenCook for Windows

Version 1.0.0 (Initial Release)

by Steven Seagondollar, DropShock Digital LLC

---

Release Date: (Insert Actual Release Date, e.g., November 10, 2023)

🚀 Download LMTokenCook v1.0.0 for Windows

• Download LMTokenCook_Installer.exe (Windows Installer)
  (This link should point directly to the LMTokenCook_Installer.exe asset included in this GitHub Release.)

---

👋 Welcome to LMTokenCook for Windows!

This is the first official release of LMTokenCook for the Windows platform. LMTokenCook is a powerful utility designed for AI power-users, developers, researchers, and writers who need to prepare large amounts of text-based content for use with Large Language Models (LLMs) that have web-based interfaces. It helps you consolidate various file types and intelligently divide them into manageable "servings" based on token limits.

This Windows version provides the full functionality of LMTokenCook through its intuitive Graphical User Interface (GUI), packaged as a convenient .exe installer.

For a complete understanding of LMTokenCook's capabilities, all its features, detailed usage instructions (including the GUI walkthrough and how to set optimal serving sizes), advanced configuration options, troubleshooting advice, and more, please consult the main project README.md file. This release note focuses specifically on getting started with this Windows installer package.

---

🪟 Windows Installation & First Launch Instructions

Follow these steps to install and run LMTokenCook on your Windows PC:

1. Download the Installer:

   • Click the download link above to get the LMTokenCook_Installer.exe file. Save it to a convenient location on your computer, such as your Downloads folder.

2. Run the Installer:

   • Locate the downloaded LMTokenCook_Installer.exe file and double-click it to start the installation wizard.
   • Windows Defender SmartScreen Prompt (Important): When you run the installer, Windows Defender SmartScreen may display a blue prompt stating "Windows protected your PC." This is a standard security measure for new applications from independent developers that haven't yet built up a widespread reputation with Microsoft, especially if the installer is not yet digitally signed with an expensive Extended Validation (EV) certificate.
     1. In the SmartScreen dialog, click on the "More info" link (it's usually a small text link).
     2. After clicking "More info," the dialog will often expand to show more details, including the application name (e.g., "LMTokenCook").
     3. A button labeled "Run anyway" should then appear. Click this button to allow the LMTokenCook installer to proceed.
   • Follow the on-screen instructions provided by the installer wizard. This typically involves steps like:
     • Accepting the software license agreement (MIT License).
     • Choosing your preferred installation location on your hard drive (the default location is usually appropriate for most users).
     • Deciding whether to create Desktop shortcuts or Start Menu entries for easy access to LMTokenCook after installation.

3. Launch LMTokenCook:

   • Once the installation process is complete, you can launch LMTokenCook. You can usually do this from your Windows Start Menu (it should be listed under recently installed programs or within its own "LMTokenCook" or "DropShock Digital" folder, depending on how the installer is configured) or by using the Desktop shortcut if one was created during installation.

4. Using LMTokenCook:
   LMTokenCook is now installed and ready for use! Launch the application. For guidance on its features and how to use the interface effectively, please see the "Usage Guide" section in the main README.md.

   

   The LMTokenCook interface on Windows.

---

✨ Windows Version Highlights (v1.0.0)

This initial Windows release of LMTokenCook delivers the core suite of tools designed to help you prepare and segment large text-based datasets for LLMs:

• Native Windows Application: A self-contained .exe installer for straightforward setup and use on Windows systems, built with CustomTkinter.
• Full GUI Functionality: Access all LMTokenCook features through its user-friendly graphical interface, including:
  • Selection of input files or folders (via browse or drag-and-drop).
  • Choice of output parent directory.
  • Setting the crucial "Serving Size (Max Tokens)".
  • Options for keeping the master file, adding line numbers, and skipping empty lines.
• Intelligent File Processing: Supports text extraction from various formats including .txt, .md, code files, .docx, .pdf (text layer), and .ipynb.
• Accurate Tokenization: Uses OpenAI's tiktoken (cl100k_base) for reliable token counting.
• Organized Output: Creates timestamped output folders with a detailed manifest.json and your processed text file(s) (masterfile.txt and/or serving_XXX_of_YYY.txt).
• Windows File Dialogs: Utilizes standard Windows open/save file dialogs for a familiar and consistent user experience.

(For a complete list of all application features and their detailed descriptions, please consult the "Key Features" section in the main README.md.)

---

🛠️ System Requirements for Windows

To ensure LMTokenCook runs effectively on your Windows PC, please make sure your system meets the following minimum requirements:

• Operating System: Windows 10 (64-bit recommended) or Windows 11.
  • While it might function on older 32-bit systems or earlier Windows versions, 64-bit Windows 10/11 is the primary target for development and testing, offering better performance and memory management.
• Processor: A 1 GHz or faster processor is required. A 2 GHz or faster multi-core processor is recommended for smoother performance, especially with large jobs.
• RAM (Memory): A minimum of 4GB of RAM is required. 8GB or more is recommended for better performance, especially when processing very large files or numerous documents.
• Hard Disk Space: Approximately 150-250MB of free disk space is required for the application installation itself. Sufficient additional disk space will be needed for your input files and the text output generated by LMTokenCook.
• Display: A screen resolution of 1280x800 pixels or higher is recommended for optimal viewing and usability of the application interface.
• .NET Framework (Typically Not Required for User): LMTokenCook is a Python application. While some Windows system components might rely on .NET, LMTokenCook itself should not require users to manually install specific .NET Framework versions, as its dependencies are primarily Python-based and handled by the application packaging.

---

⚠️ Known Considerations for This Windows Release (v1.0.0)

• Windows Defender SmartScreen Warning: As this is an initial release, the LMTokenCook installer (.exe) may not yet be code-signed with a Microsoft-recognized Extended Validation (EV) certificate. Because of this, you will likely encounter the Windows Defender SmartScreen prompt during installation, warning that "Windows protected your PC." Please follow the instructions provided in the "Windows Installation & First Launch Instructions" section above (click "More info," then "Run anyway") to proceed with the installation. This is a common occurrence for new software from independent developers.
• Antivirus Software Flags (Potential): Some aggressive antivirus programs might flag the LMTokenCook installer or the installed application executable as unknown or potentially suspicious, simply because it's a new application that their virus definition databases haven't widely encountered yet. This is usually a false positive. If your antivirus interferes with installation or execution, you may need to temporarily disable its real-time scanning or add an exception (whitelist) for the LMTokenCook installer and its installation directory. Remember to exercise caution and re-enable your antivirus afterward.
• PDF Text Extraction Limitations: Text extraction from PDF files relies on a proper text layer. LMTokenCook does not perform OCR on image-only PDFs. The quality of extraction can also vary with PDF complexity.
• Drag-and-Drop: While supported, if you encounter any issues with drag-and-drop functionality, please use the "Browse..." buttons as the primary method for selecting files/folders.

---

🛟 Support & Feedback for Windows Version

If you encounter issues specific to this Windows version of LMTokenCook, or have feedback related to its performance, installation,...