Category: Blog

  • Datathon-Health-Risk-Prediction

    Heath Risk Prediction for SCOR Datathon

    This repository is for SCOR Datathon held from November 2019 to February 2020 in Paris.
    SCOR is a tier 1 reinsurance company in the world. During this 4 months, we processed real open data, NHANES and NHCS, and built models to predict health risks of a person in U.S.

    Our business problem identification

    We have identified 2 biggest business problems for insurance and re-insurance companies; frauds and increase cost from chronic diseases.
    With the growth of middle class and urbanisation, sedentary lifestyle is becoming more common across the world. This leads to an accerelation of costly chronic diseases.
    Health care payers are seeking for solutions to decrease their premium cost not only by predicting one’s health risks to set correct premium, but also by improving one’s health; the longer the people without health problems, the cheaper the health cost that the payers pays.

    Our solutions

    We have created a flask application to predict one’s diabetes risk, level of glycohemoglobin and cholesterol, and suggest how one can decrease the risk to specific level. For example, we can say, to decrease the risk to 50% to 20% , we can suggest the person to walk 10 minutes more or sleep 1 hour longer per day.


    Main Models for our application

    • Cox.PH to identify key features for health risks
    • Kaplan Meier to visualize survival curve for each indicators
    • Gradient Boost Regression to predict Glycohemoglobin and Cholesterol level of a person
    • Xgboost Classification to predict diabetes
    • SHAP value to calculate the marginal effect of certain variables to diabetes
    • Random Survival Forest to identify the probability change considering given age and other parameter change.
    Visit original content creator repository https://github.com/cnai-ds/Datathon-Health-Risk-Prediction
  • freezeralarm

    freezeralarm

    pi zero w based freezer alarm with sms

    Easy way to install the dependencies and freezeralarm repo
    curl https://raw.githubusercontent.com/matt-desmarais/freezeralarm/master/install.sh | bash

    Configuration script
    bash /home/pi/freezeralarm/configure.sh
    Paste in Twilio and Adafruit credentials when prompted

    Single Door Alarm
    Use singledooralarm.py
    Connect the buzzer
    Connect Pi 3.3V to Buzzer +
    Connect Pi Pin 17 to Buzzer –
    Connect Pi Ground to Buzzer N

    Connect the door sensor, doesn’t matter which wire goes where
    attach one wire to Pi Ground
    attach the other wire to Pi Pin 18

    Connect the LED
    attach Pi Ground to black wire
    attach Pi Pin 16 to the red wire
    attach Pi Pin 20 to the green wire
    attach Pi Pin 21 to the blue wire
    wiringdiagramsingledooor

    Double Door Alarm
    Use doubledooralarm.py
    Connect the buzzer
    Connect Pi 3.3V to Buzzer +
    Connect Pi Pin 17 to Buzzer –
    Connect Pi Ground to Buzzer N

    Connect the door sensors, doesn’t matter which wire goes where
    Attach the first door sensor
    attach one wire to Pi Ground
    attach the other wire to Pi Pin 18
    Attach the second door sensor
    attach one wire to Pi Ground
    attach the other wire to Pi Pin 27

    Connect the LEDs
    attach Pi Ground to black wire
    attach Pi Pin 16 to the red wire
    attach Pi Pin 20 to the green wire
    attach Pi Pin 21 to the blue wire

    attach Pi Ground to black wire
    attach Pi Pin 13 to the red wire
    attach Pi Pin 19 to the green wire
    attach Pi Pin 26 to the blue wire
    wiringdiagramdoubledooor

    Temperature Monitoring
    Use tempiodata.py
    Connect the MAX31855
    Connect Pi 3.3V to MAX31855 Vin
    Connect Pi GND to MAX31855 GND
    Connect Pi GPIO 10 to MAX31855 DO.
    Connect Pi GPIO 9 to MAX31855 CS.
    Connect Pi GPIO 11 to MAX31855 CLK.
    wiringdiagramMAX31855

    Parts List:
    Pi Zero W – $10
    Header pins – $1
    F/F jumper wires 3″ – $4
    F/F jumper wires 12″ – $8
    Squid LEDs – $10 (2 count) or Squid LED – $4 (single)
    Active Piezo Buzzer – $5 (5 count)
    Magnetic Door Switch(s) – $4 each
    MAX31855 – $15
    Thermocouple – $10

    Visit original content creator repository https://github.com/matt-desmarais/freezeralarm
  • MSDB

    MSDB

    An efficient array storage engine for managing multi-dimensional arrays.

    The MSDB software provides various compression options to make the array compact, and it also can fastly perform queries on them.
    This library can be embedded any C++ projects.
    It adapts Array Functional Language (AFL), which is widely used in many array databases, instead of SQL.

    Prototype version repository

    https://github.com/RonyK/SEACOW

    The code forked from the repository

    (372 commits, ~Sep 9, 2021)

    Build

    MSDB has been developed on Windows 10 with Visual studio 2019

    Build Requirements


    Query

    You can query an array with C++ API, which provides AFL like operators.

    Build operator

    • Example query for building two-dimensional array.

    msdb::Build(
        0, msdb::dummy::star1024x1024::arrName,
        {
            msdb::DefDimension("Y", 0, 1024, 128, 32),
            msdb::DefDimension("X", 0, 1024, 128, 32)
        },
        {
            msdb::DefAttribute("ATTR_1", msdb::core::concreteTy<uint8_t>())
        }
    );

    Between operator

    • Example query for range selection on two-dimensional array.

    msdb::Between(
        msdb::Load(msdb::Array(ctx, msdb::dummy::star1024x1024::arrName)),
        msdb::Domain(msdb::Coordinate({ 100, 100 }), msdb::Coordinate({ 104, 104 }))
    );

    Filter operator

    • Example query for filter out value on two-dimensional array

    msdb::Filter(
       msdb::Load(msdb::Array(ctx, msdb::dummy::star1024x1024::arrName)),
       msdb::Attribute("ATTR_1") >= 7
    );

    Visit original content creator repository
    https://github.com/KUDB/MSDB

  • segmentation_metrics

    Segmentaion Metrics Package DOI

    GitHub release (latest SemVer) publish workflow status codecov test workflow status Documentation Status OSCS Status

    This is a simple package to compute different metrics for Medical image segmentation(images with suffix .mhd, .mha, .nii, .nii.gz or .nrrd ), and write them to csv file.

    BTW, if you need the support for more suffix, just let me know by creating new issues

    Summary

    To assess the segmentation performance, there are several different methods. Two main methods are volume-based metrics and distance-based metrics.

    Metrics included

    This library computes the following performance metrics for segmentation:

    Voxel based metrics

    • Dice (F-1)
    • Jaccard
    • Precision
    • Recall
    • False positive rate
    • False negtive rate
    • Volume similarity

    The equations for these metrics can be seen in the wikipedia.

    Surface Distance based metrics (with spacing as default)

    • Hausdorff distance
    • Hausdorff distance 95% percentile
    • Mean (Average) surface distance
    • Median surface distance
    • Std surface distance

    Note: These metrics are symmetric, which means the distance from A to B is the same as the distance from B to A. More detailed explanication of these surface distance based metrics could be found here.

    Installation

    $ pip install seg-metrics

    Getting started

    Tutorial is at the Colab.

    API reference is available at Documentation

    Examples could be found below.

    Usage

    At first, import the package:

    import seg_metrics.seg_metrics as sg

    Evaluate two batches of images with same filenames from two different folders

    labels = [0, 4, 5 ,6 ,7 , 8]
    gdth_path = 'data/gdth'  # this folder saves a batch of ground truth images
    pred_path = 'data/pred'  # this folder saves the same number of prediction images
    csv_file = 'metrics.csv'  # results will be saved to this file and prented on terminal as well. If not set, results 
    # will only be shown on terminal.
    
    metrics = sg.write_metrics(labels=labels[1:],  # exclude background
                      gdth_path=gdth_path,
                      pred_path=pred_path,
                      csv_file=csv_file)
    print(metrics)  # a list of dictionaries which includes the metrics for each pair of image.

    After runing the above codes, you can get a list of dictionaries metrics which contains all the metrics. Also you can find a .csv file containing all metrics in the same directory. If the csv_file is not given, the metrics results will not be saved to disk.

    Evaluate two images

    labels = [0, 4, 5 ,6 ,7 , 8]
    gdth_file = 'data/gdth.mhd'  # ground truth image full path
    pred_file = 'data/pred.mhd'  # prediction image full path
    csv_file = 'metrics.csv'
    
    metrics = sg.write_metrics(labels=labels[1:],  # exclude background
                      gdth_path=gdth_file,
                      pred_path=pred_file,
                      csv_file=csv_file)

    After runing the above codes, you can get a dictionary metrics which contains all the metrics. Also you can find a .csv file containing all metrics in the same directory.

    Note:

    1. When evaluating one image, the returned metrics is a dictionary.
    2. When evaluating a batch of images, the returned metrics is a list of dictionaries.

    Evaluate two images with specific metrics

    labels = [0, 4, 5 ,6 ,7 , 8]
    gdth_file = 'data/gdth.mhd'
    pred_file = 'data/pred.mhd'
    csv_file = 'metrics.csv'
    
    metrics = sg.write_metrics(labels=labels[1:],  # exclude background if needed
                      gdth_path=gdth_file,
                      pred_path=pred_file,
                      csv_file=csv_file,
                      metrics=['dice', 'hd'])
    # for only one metric
    metrics = sg.write_metrics(labels=labels[1:],  # exclude background if needed
                      gdth_path=gdth_file,
                      pred_path=pred_file,
                      csv_file=csv_file,
                      metrics='msd')  

    By passing the following parameters to select specific metrics.

    - dice:         Dice (F-1)
    - jaccard:      Jaccard
    - precision:    Precision
    - recall:       Recall
    - fpr:          False positive rate
    - fnr:          False negtive rate
    - vs:           Volume similarity
    
    - hd:           Hausdorff distance
    - hd95:         Hausdorff distance 95% percentile
    - msd:          Mean (Average) surface distance
    - mdsd:         Median surface distance
    - stdsd:        Std surface distance

    For example:

    labels = [1]
    gdth_file = 'data/gdth.mhd'
    pred_file = 'data/pred.mhd'
    csv_file = 'metrics.csv'
    
    metrics = sg.write_metrics(labels, gdth_file, pred_file, csv_file, metrics=['dice', 'hd95'])
    dice = metrics['dice']
    hd95 = metrics['hd95']

    Evaluate two images in memory instead of disk

    Note:

    1. The two images must be both numpy.ndarray or SimpleITK.Image.
    2. Input arguments are different. Please use gdth_img and pred_img instead of gdth_path and pred_path.
    3. If evaluating numpy.ndarray, the default spacing for all dimensions would be 1.0 for distance based metrics.
    4. If you want to evaluate numpy.ndarray with specific spacing, pass a sequence with the length of image dimension as spacing.
    labels = [0, 1, 2]
    gdth_img = np.array([[0,0,1], 
                         [0,1,2]])
    pred_img = np.array([[0,0,1], 
                         [0,2,2]])
    csv_file = 'metrics.csv'
    spacing = [1, 2]
    metrics = sg.write_metrics(labels=labels[1:],  # exclude background if needed
                      gdth_img=gdth_img,
                      pred_img=pred_img,
                      csv_file=csv_file,
                      spacing=spacing,
                      metrics=['dice', 'hd'])
    # for only one metrics
    metrics = sg.write_metrics(labels=labels[1:],  # exclude background if needed
                      gdth_img=gdth_img,
                      pred_img=pred_img,
                      csv_file=csv_file,
                      spacing=spacing,
                      metrics='msd')  

    About the calculation of surface distance

    The default surface distance is calculated based on fullyConnected border. To change the default connected type, you can set argument fullyConnected as False as follows.

    metrics = sg.write_metrics(labels=[1,2,3],
                            gdth_img=gdth_img,
                            pred_img=pred_img,
                            csv_file=csv_file,
                            fully_connected=False) 

    In 2D image, fullyconnected means 8 neighbor points, while faceconnected means 4 neighbor points. In 3D image, fullyconnected means 26 neighbor points, while faceconnected means 6 neighbor points.

    How to obtain more metrics? like “False omission rate” or “Accuracy”?

    A great number of different metrics, like “False omission rate” or “Accuracy”, could be derived from some the confusion matrics. To calculate more metrics or design custom metrics, use TPTNFPFN=True to return the number of voxels/pixels of true positive (TP), true negative (TN), false positive (FP), false negative (FN) predictions. For example,

    metrics = sg.write_metrics(
                            gdth_img=gdth_img,
                            pred_img=pred_img,
                            TPTNFPFN=True) 
    tp, tn, fp, fn = metrics['TP'], metrics['TN'], metrics['FP'], metrics['FN']
    false_omission_rate = fn/(fn+tn)
    accuracy = (tp + tn)/(tp + tn + fp + fn)

    Comparision with medpy

    medpy also provide functions to calculate metrics for medical images. But seg-metrics
    has several advantages.

    1. Faster. seg-metrics is 10 times faster calculating distance based metrics. This jupyter notebook could reproduce the results.
    2. More convenient. seg-metrics can calculate all different metrics in once in one function while medpy needs to call different functions multiple times which cost more time and code.
    3. More Powerful. seg-metrics can calculate multi-label segmentation metrics and save results to .csv file in good manner, but medpy only provides binary segmentation metrics. Comparision can be found in this jupyter notebook.

    If this repository helps you in anyway, show your love ❤️ by putting a ⭐ on this project. I would also appreciate it if you cite the package in your publication. (Note: This package is NOT approved for clinical use and is intended for research use only. )

    Citation

    If you use this software anywhere we would appreciate if you cite the following articles:

    Jia, Jingnan, Marius Staring, and Berend C. Stoel. “seg-metrics: a Python package to compute segmentation metrics.” medRxiv (2024): 2024-02.

    @article{jia2024seg,
      title={seg-metrics: a Python package to compute segmentation metrics},
      author={Jia, Jingnan and Staring, Marius and Stoel, Berend C},
      journal={medRxiv},
      pages={2024--02},
      year={2024},
      publisher={Cold Spring Harbor Laboratory Press}
    }
    
    Visit original content creator repository https://github.com/Jingnan-Jia/segmentation_metrics
  • Data_structures_sentinel_dynamic_link

    Data_structures_sentinel_dynamic_link

    This repository will contain three different types of data structures:

    1. Sentinel-based data structure
    2. Dynamic data structure using malloc for memory allocation
    3. Linked data structure using pointers to point to the next element in the structure

    Sentinel-based Data Structure

    The sentinel-based data structure is a type of data structure that uses a sentinel node to mark the beginning and end of the structure. The sentinel node is a special node that does not contain any actual data but is used as a marker to indicate the beginning or end of the structure. This type of data structure is often used in algorithms that need to search through a data structure, such as sorting algorithms or binary search.

    Dynamic Data Structure using malloc

    The dynamic data structure is a type of data structure that uses malloc to allocate memory dynamically. This means that the size of the data structure can be determined at run-time rather than at compile-time. This type of data structure is often used in situations where the size of the data to be stored is not known in advance.

    Linked Data Structure using Pointers

    The linked data structure is a type of data structure that uses pointers to link each element to the next element in the structure. Each element contains a data field and a pointer field that points to the next element in the structure. This type of data structure is often used in situations where the data needs to be accessed sequentially, such as in a queue or a linked list.

    Visit original content creator repository
    https://github.com/NotTheStallion/Data_structures_sentinel_dynamic_link

  • hafele_connect_mesh

    Häfele Connect Mesh Integration for Home Assistant (HACS)

    Add Häfele mesh devices to Home Assistant

    This repository provides an integration for Home Assistant (HACS) to allow Häfele Connect Mesh devices to be controlled within Home Assistant.

    Requirements

    Setup Guide

    Step 1: Set Up Häfele Connect Mesh on Your Phone

    Most users will have already completed this step. If you haven’t, or if you encounter any issues, follow these instructions:

    1. Download the Häfele Connect Mesh App:

    2. App Configuration:

      • Open the app and go to Settings.
      • Create or sign in to your account.
      • Click on ‘Set up mode’.
    3. Add Devices:

      • Go to Dashboard > Devices > + icon.
      • When the connect-mesh gateway is powered on and connected to the internet, it will show in the discovery list. Add the device.

      Note: If the last step doesn’t work, the device may still be connected to your phone. Switch back to ‘Control mode’ on the dashboard, and a blue popup should appear allowing you to connect to the ‘cloud’.

    4. Ensure Other Devices Are Connected:

      • Make sure you have other devices connected that you will need to control through Home Assistant.
    5. Upload App Configuration to Cloud:

      After making any changes in the app or completing the first setup:

      1. Change to ‘Control mode’ in the app.
      2. Navigate to Gateway > Cloud settings > Synced networks.
      3. Click on the network upload button to sync your configuration with the cloud.

    Troubleshooting

    • Lights on the Gateway:
      • BLE flickering: Not connected to the Connect Mesh app
        • Reset the gateway device with the physical button and try again.
      • Internet flickering: Not connected to the internet.
        • Check if the cable works and you are signed into an account on the app.

    Step 2: Create an API Token

    1. Make sure you signed into your Connect Cloud Account and ensure all gateway lights are on.
    2. Go to Connect Mesh Cloud and sign in with the account you created in Step 1.
    3. Navigate to the Developer Page (use this link as there’s no button).
    4. Create a new API token:
      • Use the offset to change the expiration date (unit: MONTH, offset: 36, for 3 years).
      • Click on SET before creating the token.
      • Example token: CMC_ab12cd34_ef56gh78ij90kl12mn34op56qr78st90uv12wx34yz56ab78cd90ef12.

    Step 3: Add Custom Repository in HACS

    1. In Home Assistant Community Store (HACS), add this custom repository.
    2. Install Häfele Connect Mesh.

    Step 4: Add the Integration

    1. Add the Häfele Connect Mesh integration.
    2. Fill in the API token.
    3. Select the network you want to add and submit.

    You are all set!

    To Do List

    • Add a way to change RGB color when API supports this functionality.
    • Resolve the issue where name changes through the app don’t appear in Home Assistant.
    • Investigate the possibility to see states when using a physical button.

    Feel free to customize it further as per your needs.

    Visit original content creator repository
    https://github.com/guillaumeseur/hafele_connect_mesh

  • volleyball-ml

    volleyball-ml

    Usage

    Example to generate the data and train the model for the year 2019. Run the code in the root of the project folder :

    python src/data_collection/ncaa_team_scraper.py -y 2019
    python src/data_collection/ncaa_player_scraper.py -y 2019
    python src/data_collection/ncaa_team_data_cleaner.py -y 2019 --all
    python src/data_collection/ncaa_player_data_cleaner.py -y 2019 --all
    python src/models/ncaa_model_evaluator.py -d "../../data/ncaa/processed/2019/accumulated/0.2_ewm_with_players.csv"
    python src/models/ncaa_model_tuner.py -d "../../data/ncaa/processed/2019/accumulated/0.2_ewm_with_players.csv"

    Description of relevant scripts

    Final Scripts

    ncaa_team_scraper.py :

    • Scrapes game by game data and team statistics for all the teams
    • Data location : data/ncaa/raw/
    ncaa_player_scraper.py :

    • Scrapes individual player performance data
    • Data location : data/ncaa/raw
    ncaa_team_data_cleaner.py :

    • Cleans and preprocesses team data and creates team_vs_team (without players stats) dataset s
    • Data location : data/ncaa/processed/{year}, data/ncaa/processed/{year}/accumulated
    ncaa_team_player_cleaner.py :

    • Cleans and preprocesses team data and creates team_vs_team (with player stats) dataset
    • Data location : data/ncaa/processed/{year}, data/ncaa/processed/{year}/accumulated
    ncaa_model_evaluator.py :

    • Evaluates five different machine learning models on the dataset present at the entered path
    ncaa_model_tuner.py :

    • Experiments with tuning and feature selection on the dataset and different models

    Other scripts

    ncaa_combine_data(deprecated).py

    • Generates a dataset with data compiled over all the years
    clean_team_stats_filenames.py

    • Cleans the filenames in the data/ncaa/raw/{year}/team_stats directory
    • Needed before using ncaa_combine_data(deprecated).py
    src/models/experiments

    • Random experiments with different data, models and techniques

    Contributors

    Visit original content creator repository
    https://github.com/threewisemonkeys-as/volleyball-ml

  • expressive_sort

    expressive sort

    In this quora answere I wrote about javascript/python expressiveness vs Go/Rust/D/… performance.

    As an example, I mentioned the “3 lines” haskell quick sort and wrote this javascript version

    const sorted = ([pivot, ...others]) => 
      pivot === void 0 ? [] : [
        ...sorted(others.filter(n => n < pivot)),
        pivot,
        ...sorted(others.filter(n => n >= pivot))
      ];

    This, of course, is not a “quick sort” because the original one is an “in place” algorithm that doesn’t require additional memory space allocation. This is a functional oriented expression that exemplarizes how expressive a “functional” orientation can be (You “express” that the sorted version of an array is, given one of it’s elements, the sorted version of the smaller ones, plus the item, plus the sorted version of the bigger ones).

    As an enthusiastic newbie to the “D” programming language, I thought that D could affort this expressiveness too…

    D has no support for destructuring as javascript has (remember de sorted([pivot, ...others])), but it has lambdas, map/filter/reduce support, array slices and array concatenation that allows you to write easily a similar expression:

    T[] sorted(T)(T[] xs)
    {
      return xs.length == 0 ? [] : 
        xs[1..$].filter!(x=> x < xs[0]).array.sorted ~ 
        xs[0..1] ~ 
        xs[1..$].filter!(x=> x >= xs[0]).array.sorted;
    }

    note: D notation allows to write foo(a) as a.foo() or a.foo, this is the reason we can write sorted( array( something ) ) as something.array.sorted

    note: sorted is a templated method (T is the type of the elements of the array): “under the scenes”, D compiler detects if the final used type is comparable (i.e.: it is a class with a opCmp method, or it is a numerical type, or …): yo don’t need to tell that T extends something like “IComparable” because D libraries are not “interface” based: D prefers to use “conventions” and check them using instrospection at compile time (D developers write compile-time code and run-time code at the same time: D allows you to mix them naturally).

    Seeing the similarities, I assume (I really don’t know) that javascript and D versions are doing the same “under the scenes”:

    • The [...array1, ...array2, ...array3] javascript is equivalent to the array1 ~ array2 ~ array3 D code. That is, a new array is being generated as a result of copying the elements of the original 3.
    • The .filter!(...).array D code is using a “Range” to filter the elements and the “.array()” method to materialize the selected elements as an array. Internally, it is similar to the javascript code where .filter(...) iterates and selects the resulting elements and finally materializes the array

    Wich one will perform better?

    First surprise was Javascript (nodejs) version performs better than D (about 20% faster for 1_000_000 random Float64 numbers).

    • Javascript (node): 1610 ms
    • D (DMD compiler): 2017 ms

    Fortunately, D has 3 compilers: DMD (official reference compiler), GDC (GCC based compiler) and LDC (LLVM based compiler).

    • D (LDC compiler): 693 ms !!!

    That’s speed 🙂

    After some tests, I realized javascript destructuring [pivot,…others] caused a performance penalty and I rewrote to a more efficient version (very similar to D syntax)

    const sorted = (xs) => 
      xs.length===0 ? [] : [].concat(
        sorted(xs.slice(1).filter(x => x < xs[0])),
        xs[0],
        sorted(xs.slice(1).filter(x => x >= xs[0]))
      );

    And the execution on 1 million floats reduced to 921 ms!!! That’s impressive :-O

    I decided to write similar code in other languajes to compare.

    In Python:

    def sorted(xs):
      return [] if len(xs) == 0 else \
        sorted([x for x in xs[1:] if x < xs[0]]) + \
        xs[0:1] + \
        sorted([x for x in xs[1:] if x >= xs[0]])

    In Crystal:

    def sorted(xs : Array(Float64)) : Array(Float64)
      return xs.size == 0 ? [] of Float64  :
        sorted(x[1..].select { |x| x < xs[0] }) +
        [ xs[0] ] +
        sorted(xs[1..].select { |x| x >= xs[0] })
    end

    In Julia (thanks to Camilo Chacón Sartori)

    function sorted(xs::Array{Float64, 1})::Array{Float64, 1}
      return length(xs) == 0 ? Array{Float64, 1}() : vcat(
        sorted([x for x in xs[2:end] if x < xs[1]]),
        xs[1],
        sorted([x for x in xs[2:end] if x >= xs[1]])
      )
    end

    In Scala

    def sorted( xs:List[Double] ): List[Double] =
      xs match {
        case Nil =>
          List()
        case pivot :: rest =>
          sorted( rest.filter( _ < pivot ) ) ++
          List(pivot) ++
          sorted( rest.filter( _ >= pivot ) )
      }

    In Nim

    func sorted[T](xs:seq[T]): seq[T] = 
      if xs.len==0: @[] else: concat(
        xs[1..^1].filter(x=>x<xs[0]).sorted,
        @[xs[0]],
        xs[1..^1].filter(x=>x>=xs[0]).sorted
      )

    For better measurement:

    • each test is internally ran 5 times, each time with a newly generated set of numbers (to avoid run-to-run optimization effects).
    • Between compilers (or interpreters) tests, there is a pause of 30s to normalize system (Mainly CPU temperature)

    The results, as CSV, are

    compiler,lang,size,ms
    "ldc2","D",1000000,645
    "ldc2","D",1500000,978
    "ldc2","D",3000000,2013
    "ldc2","D",6000000,4165
    "crystal","Crystal",1000000,713
    "crystal","Crystal",1500000,1102
    "crystal","Crystal",3000000,2277
    "crystal","Crystal",6000000,4750
    "node","Javascript",1000000,904
    "node","Javascript",1500000,1384
    "node","Javascript",3000000,2977
    "node","Javascript",6000000,6117
    "nim","Nim",1000000,1038
    "nim","Nim",1500000,1564
    "nim","Nim",3000000,3352
    "nim","Nim",6000000,6951
    "scala","Scala",1000000,1524
    "scala","Scala",1500000,2537
    "scala","Scala",3000000,5741
    "scala","Scala",6000000,19819
    "dmd","D",1000000,1818
    "dmd","D",1500000,2773
    "dmd","D",3000000,5822
    "dmd","D",6000000,12123
    "julia","julia",1000000,3224
    "julia","julia",1500000,4648
    "julia","julia",3000000,9429
    "julia","julia",6000000,20264
    "python3","Python",1000000,4512
    "python3","Python",1500000,7339
    "python3","Python",3000000,17428
    "python3","Python",6000000,38981
    

    Execution time histogram by array size:

    Process time

    Changing to logarithmic scale

    Process time

    Do you know how to improve?

    I include the code to the 7 tests. Please, tell me if you see something we can improve:

    • Avoid imperative instructions: “sorted” must be an unique expression or, at least, an unique “return …” statement funcion.
    • Of course, you can’t use built-in library sort methods 🙂
    • Remember that this is not a quick-sort performance test (that, obviously, can be implemented in a more efficient way)

    Running the tests

    Prerequisites

    All tests have been executed on a Ubuntu 20.04 linux.

    Tests require Nodejs, Python3, Julia, DMD/LDC for D, scala, Nim and Crystal compilers

    • Nodejs: I use node 12.20 (see NodeSource distributions for more information)
    • Python3: Ubuntu comes with python 3 preinstalled.
    • Julia: It is available in apt and snap repositories. The apt version is newest (but not latest). See download page for newest versions
    $ sudo apt install julia
    • DMD and LDC: They are available in apt and snap repositories (see guide ) . Because ldc version is newest in snap, I use snap repos:
    $ snap install dmd --classic
    $ snap install ldc2 --classic
    • scala 2.11.12: It is availabe in apt repository
    $ sudo apt install scala
    • Crystal: It is availabe in apt and snap repositories (see guide for more information )

    Running

    You can run all tests using test.sh

    $ ./test.sh 

    If no want to generate a result.csv file

    $ ./test.sh | tee result.csv

    If you prefer to run tests individually:

    • D (LDC)
    $ ldc2 -O5 -release -enable-cross-module-inlining --run sorted.d 
    • D (DMD)
    $ dmd -O -release -run sorted.d
    • Javascript
    $ node sorted.js
    • Crystal
    $ crystal run sorted.cr --release
    • Python
    $ python3 -OO sorted.py
    • Julia
    julia -O3 --inline=yes --check-bounds=no sorted.jl

    thanks to

    Visit original content creator repository https://github.com/ddcovery/expressive_sort
  • ub-bint

    ub-bint

    Last Commit wakatime GitHub

    This repository serves as a documentation of my journey through UB’s Information Technology program. It’s a practical, hands-on account of my experiences in coding and IT projects. From tackling coding challenges to exploring various programming languages, it’s a raw and unfiltered view of my growth as a programmer. No frills, just real progress.

    Table of Contents

    About

    This repository serves to chronicle my experiences and projects during my time in UB’s Information Technology program. It showcases my progress as a programmer, from coding challenges to complex IT projects.

    File Structure

    Folders are divided by semester, course, and assessment type.

    • 2023-1 Fall Semester (August 2023 to December 2023)
      • [cmps1131] Principles of Programming 1
      • [cmps1134] Fundamentals of Computing
    • 2023-2 Spring Semester (January 2024 to May 2024)
      • [cmps1232] Principles of Programming 2
    • 2024-1 Fall Semester (August 2024 to December 2024)
    • 2024-2 Spring Semester (January 2025 to May 2025)
    • 2025-1 Fall Semester (August 2025 to December 2025)
    • 2025-2 Spring Semester (January 2026 to May 2026)
    • 2026-1 Fall Semester (August 2026 to December 2026)
    • 2026-2 Spring Semester (January 2027 to May 2027)

    Navigating Semesters, Courses, and Assessments

    This repository is organized by semesters, courses, and assessment type. Here’s how you can navigate through the structure:

    • 20XX-X: Click on the respective semester folder to explore the courses undertaken during that semester.
    • Course Name: Inside each semester folder, find individual course folders. Each course folder contains assignments and assessments.
    • Assessment Type: Inside each course folder, explore different types of assessment folders. These folders contain project files, assignments, and other materials related to the coursework.

    License

    This project is licensed under the MIT License – see the LICENSE file for details.

    Visit original content creator repository https://github.com/jennxsierra/ub-bint
  • Image-classifier

    Supervised Learning (Image classifier)

    using TensorFlow and Keras

    This project is part of the CharityML project for the Udacity course: Intro to Machine Learning with TensorFlow, and its github repository.

    In this project, first a code for an image classifier is built with TensorFlow and a Keras model is generated, then it will be converted it into a command line application. the command line application takes an image and the trained Keras model, and then returns the top K most likely class labels along with the probabilities.

    Goal

    Classify images of flowers to 102 diffrent categories.

    Data

    A dataset from Oxford of 102 flower categories is used. This dataset has 1,020 images in the training and avaluation set, and 6,149 images in the test set.

    Model

    • A MobileNet pre-trained network from TensorFlow Hub is used.
      The model is used without its last layer, and some fully connected layers with dropout is added sequentially to be trained on the data.

    • A model using Conv2D and some fully connected layers with dropout is added sequentially to be trained on the data.

    Code

    The code is provided in the finding_donors.ipynb notebook file. the visuals.py Python file and the census.csv dataset file is used.

    Run

    In a terminal or command window, navigate to the top-level project directory finding_donors/ (that contains this README) and run one of the following commands:

    ipython notebook Project_Image_Classifier_Project.ipynb.ipynb

    or

    jupyter notebook Project_Image_Classifier_Project.ipynb.ipynb

    This will open the iPython Notebook software and project file in your browser.

    or for the python application run one of the follwoing:

    python predict.py /path/to/image best_model
    python predict.py /path/to/image best_model --top_k 5
    python predict.py /path/to/image best_model --category_names label_map.json

    It will print the top_k or 3 most probable predicted classes for the image. If the label_map.json is also given, the the name of each flower representing each class will be shown.

    Visit original content creator repository
    https://github.com/mbshbn/Image-classifier