Knowledge Base Resources
Use these resources “vetted” by the community. Additional Knowledge Base Resources are always welcome.
GIS: What is a Geodetic Datums?
0
Often when working with GIS, or spatial data, one encounters the word "datum" and it may require that you choose a "datum" when doing GIS computation tasks. Below is a short video on what are datums from NOAA and UCAR.
AI for improved HPC research - Cursor and Termius - Powerpoint
0
These slides provide an introduction on how Termius and Cursor, two new and freemium apps that use AI to perform more efficient work, can be used for faster HPC research.
DeepChem
0
DeepChem is an open-source library built on TensorFlow and PyTorch. It is helpful in applying machine learning algorithms to molecular data.
Educause HEISC-800-171 Community Group
0
The purpose of this group is to provide a forum to discuss NIST 800-171 compliance. Participants are encouraged to collaborate and share effective practices and resources that help higher education institutions prepare for and comply with the NIST 800-171 standard as it relates to Federal Student Aid (FSA), CMMC, DFARS, NIH, and NSF activities.
Running Particle-in-Cell Simulations on HPC
0
WarpX is an advanced particle-in-cell code used to model particle accelerators, which needs to be run on HPC. This website contains the tutorial on how to build WarpX on various HPC systems such as NERSC along with examples on how to set up post-processing/visualization tools for different physics cases.
File management of Visual Studio Code on clusters
0
Visual Studio Code, commonly known as VSCode, is a popular tool used by programmers worldwide. It serves as a text editor and an Integrated Development Environment (IDE) that supports a wide variety of programming languages. One of its key features is its extensive library of extensions. These extensions add on to the basic functionalities of VSCode, making coding more efficient and convenient.
However, there's a catch. When these extensions are installed and used frequently, they generate a multitude of files. These files are typically stored in a folder named .vscode-extension within your home directory. On a cluster computing facility such as the FASTER and Grace clusters at Texas A&M University, there's a limitation on how many files you can have in your home directory. For instance, the file number limit could be 10000, while the .vscode-extension directory can hold around 4000 temporary files even with just a few extensions. Thus, if the number of files in your home directory surpasses this limit due to VSCode extensions, you might face some issues. This restriction can discourage users from taking full advantage of the extensive features and extensions offered by the VSCode editor.
To overcome this, we can shift the .vscode-extension directory to the scratch space. The scratch space is another area in the cluster where you can store files and it usually has a much higher limit on the number of files compared to the home directory. We can perform this shift smoothly using a feature called symbolic links (or symlinks for short). Think of a symlink as a shortcut or a reference that points to another file or directory located somewhere else.
Here's a step-by-step guide on how to move the .vscode-extension directory to the scratch space and create a symbolic link to it in your home directory:
1. Copy the .vscode-extension directory to the scratch space: Using the cp command, you can copy the .vscode-extension directory (along with all its contents) to the scratch space. Here's how:
cp -r ~/.vscode-extension /scratch/user
Don't forget to replace /scratch/user with the actual path to your scratch directory.
2. Remove the original .vscode-extension directory: Once you've confirmed that the directory has been copied successfully to the scratch space, you can remove the original directory from your home space. You can do this using the rm command:
rm -r ~/.vscode-extension
It's important to make sure that the directory has been copied to the scratch space successfully before deleting the original.
3. Create a symbolic link in the home directory: Lastly, you'll create a symbolic link in your home directory that points to the .vscode-extension directory in the scratch space. You can do this as follows:
ln -s /scratch/user/.vscode-extension ~/.vscode-extension
By following this process, all the files generated by VSCode extensions will be stored in the scratch space. This prevents your home directory from exceeding its file limit. Now, when you access ~/.vscode-extension, the system will automatically redirect you to the directory in the scratch space, thanks to the symlink. This method ensures that you can use VSCode and its various extensions without worrying about hitting the file limit in your home directory.
OnShape Documentation
0
This contains documentation for getting started with using OnShape for CAD. OnShape cloud-hosted CAD software that lets you work with others like on a Google Doc, with the power and capabilities of any other software like Solidworks or Inventor.
Quick and Robust Data Augmentation with Albumentations Library
0
Data augmentation is a crucial step in the pipeline for image classification with deep learning. Albumentations is an extremely versatile Python library that can be used to easily augment images. Transformations include rotations, flips, downscaling, distortions, blurs, and many more.
Citation:
Buslaev A, Iglovikov VI, Khvedchenya E, Parinov A, Druzhinin M, Kalinin AA. Albumentations: Fast and Flexible Image Augmentations. Information. 2020; 11(2):125. https://doi.org/10.3390/info11020125
Optimizing Research Workflows - A Documentation of Snakemake
0
Snakemake is a powerful and versatile workflow management system that simplifies the creation, execution, and management of data analysis pipelines. It uses a user-friendly, Python-based language to define workflows, making it particularly valuable for automating and reproducibly managing complex computational tasks in research and data analysis.
FreeSurfer Tutorials
0
The official MGH / Harvard tutorial page for FreeSurfer. The FreeSurfer group has provided and designed a series of tutorials for using FreeSurfer and for getting acquainted with the concepts needed to perform its various modes of analysis and processing of MRI data. The tutorials are designed to be followed along in a terminal window where commands can be copy/pasted instead of typed.
Python Data and Viz Training (CCEP Program)
0
Singularity/Apptainer User Manuals
0
Singularity/Apptainer is a free and open-source container platform that allows users to build and run containers on high performance computing resources.
SingularityCE is the community edition of Singularity maintained by Sylabs, a company that also offers commercial Singularity products and services.
Apptainer is a fork of Singularity, maintained by the Linux foundation, a community of developers and users who are passionate about open source software.
Framework to help in scaling Machine Learning/Deep Learning/AI/NLP Models to Web Application level
0
This framework will help in scaling Machine Learning/Deep Learning/Artificial Intelligence/Natural Language Processing Models to Web Application level almost without any time.
Vulkan Support Survey across Systems
0
It's not uncommon to see beautiful visualizations in HPC center galleries, but the majority of these are either rendered off the HPC or created using programs that run on OpenGL or custom rasterization techniques. To put it simply the next generation of graphics provided by OpenGL's successor Vulkan is strangely absent in the super computing world. The aim of this survey of available resources is to determine the systems that can support Vulkan workflows and programs. This will assist users in getting past some of the first hurdles in using Vulkan in HPC contexts.
Beautiful Soup - Simple Python Web Scraping
0
This package lets you easily scrape websites and extract information based on html tags and various other metadata found in the page. It can be useful for large-scale web analysis and other tasks requiring automated data gathering.
R for Data Science
0
R for Data Science is a comprehensive resource for individuals looking to harness the power of the R programming language for data analysis, visualization, and statistical modeling. Whether you're a beginner or an experienced data scientist, this guide will help you unlock the full potential of R in the realm of data science.
OpenMP and Multithreaded Jobs in GRASS
0
Techniques and support for multithreaded geospatial data processing in GRASS.
Managing and Optimizing Your Jobs on HPC
0
An overview of tools and methods to manage and optimize jobs and HPC workflows
Termius - Modern ssh platform
0
**Termius: The Modern SSH Client for 2023**
Termius is the future-facing SSH client that's redefining remote server access in 2023. Designed for ease and efficiency, Termius offers a seamless connection experience across all devices, be it mobile or desktop. Gone are the days of re-inputting IP addresses, ports, and passwords; with Termius, one-click connectivity is the new norm.
**How Termius Elevates Remote Server Access:**
1. **One-Click Connectivity:** Save the hassle of remembering and re-entering connection details. Termius provides an immediate connection to your infrastructure with a single click.
2. **Synchronized Across Devices:** Termius ensures that your data, connection settings, and preferences are consistent across all your devices, from mobile to desktop.
3. **Unparalleled Security:** With the Cloud Vault feature, users can securely store their data in an encrypted environment, accessible only from their specific devices. Shared vaults allow for safe connection sharing within teams.
4. **AI-Powered Terminal Experience:** Advanced AI-driven autocomplete means users can input command descriptions, and Termius will swiftly convert them into accurate bash commands, simplifying and enhancing the terminal interaction.
5. **Collaborative Troubleshooting:** Share terminal sessions with teammates, facilitating cooperative problem-solving or knowledge sharing. No additional server-side installations needed.
6. **Automation and Snippets:** Streamline routine processes with the ability to save and run frequently used shell scripts. Sharing these Snippets with your team can lead to increased productivity and fewer manual errors.
7. **All-Device Compatibility:** Whether on iPad, iPhone, Android, macOS, Windows, or Linux, Termius ensures a consistent and fluid experience. The platform's synchronization capability means you're always ready to respond swiftly, irrespective of the device in use.
For professionals and businesses aiming for top-notch server access efficiency, Termius is the gold standard in 2023. Experience the revolution in SSH connectivity and optimize your workflow with Termius.
Application Fundamentals (Android)
0
The provided text discusses various aspects of Android app development fundamentals. It covers key concepts related to app components, the AndroidManifest.xml file, and app resources. Android apps are built using various components, including Activities, Services, Broadcast Receivers, and Content Providers. These components serve different purposes and have distinct lifecycles. Activities are used for user interaction, services for background tasks, broadcast receivers for system-wide event handling, and content providers for managing shared data.The AndroidManifest.xml file is essential for declaring app components, permissions, and other settings. It informs the Android system about the app's components and capabilities. For instance, it specifies the minimum API level, declares hardware and software requirements, and defines intent filters to enable components to respond to specific actions.It's crucial to declare app requirements, such as device features and minimum Android API levels, to ensure compatibility with different devices and configurations. These declarations help in filtering the app's availability on Google Play for users with compatible devices.Android apps rely on resources separate from code, including images, layouts, strings, and more. These resources are stored in various directories and can be tailored for different device configurations. Providing alternative resources allows for optimization across different languages, screen sizes, orientations, and other factors.
Understanding these fundamentals is essential for developing Android applications effectively, ensuring compatibility, and providing a consistent user experience across a wide range of devices and configurations.
Natural Language Processing with Deep Learning
0
CS244N is a renowned natural language processing course offered by Stanford University and taught by Christopher Manning. It covers a wide range of topics in NLP, including language modeling, machine translation, sentiment analysis, and more. It teaches both foundational concepts and cutting-edge research to gain a comprehensive understanding of NLP techniques and applications.
Language models and using HPC resources
0
Documentation and research based on the latest NLP text generation detection methods for 2023.
What is VPN? How It Works, Types of VPN
0
A VPN, or Virtual Private Network, is a technology that creates a secure tunnel between your device and a VPN server. This tunnel encrypts all of your traffic, making it unreadable to anyone who tries to intercept it.
Regulated Research Community of Practice
0
The daily news clearly shows the increasing threat to safety and privacy of data, personal as well as intellectual property. While the requirements such as DFARS 7012, HIPAA, and Cybersecurity Maturity Model Certification (CMMC) improve the consistency of data handling between agencies and contractors and grantees, it leaves academic institutions to figure out how to meet such requirements in a cost-effective way that fits the research and education mission of the institution. Most institutions, agencies, and companies act in isolation with one-off contract language to address data security and safeguarding concerns. Even though cybersecurity has a clear and uniform goal of protecting data, a onesize solution does not fit all academic institutions.
By supporting this community with development of a community strategic roadmap, regular discussions and workshops, and a repository of generalized and specific resources for handling regulated research programs RRCoP lowers the barrier to entry for institutions handling new regulations.
Handwritten Digits Tutorial in PyTorch
0
This tutorial is essentially the "hello world" of image recognition and feed-forward neural network (using PyTorch). Using the MNIST database (filled within images of handwritten digits), the tutorial will instruct how to build a feed-forward neural network that can recognize handwritten digits. A solid understanding of feed-forward and back-propagation is recommended.